Continuous Outcomes

Author

Josie Athens

Published

February 16, 2024

2 Confidence Intervals around the Mean

Example

The data set we will be using for this laboratory is from Bernard, GR, et al. (1997) The effects of ibuprofen on the physiology and survival of patients with sepsis, N Engl J Med 336(13): 912–918. Here is an abbreviated version of the abstract.

\(\dots\) we conducted a randomized, double-blind, placebo-controlled trial of intravenous ibuprofen\(\dots\) in 455 patients who had sepsis, defined as fever, tachycardia, tachypnea, and acute failure of at least one organ system. In the ibuprofen group, but not the placebo group, there were significant declines in [various measures including] temperature\(\dots\) However, treatment with ibuprofen did not reduce the incidence or duration of shock or the acute respiratory distress syndrome and did not significantly improve the rate of survival at 30 days (mortality, 37 percent with ibuprofen vs. 40 percent with placebo).”

bernard = rcopy(R"pubh::Bernard")
bernard |> schema
┌──────────┬────────────────────────────┬──────────────────────────────────┐
│ names    │ scitypes                   │ types                            │
├──────────┼────────────────────────────┼──────────────────────────────────┤
│ id       │ Continuous                 │ Float64                          │
│ treat    │ Multiclass{2}              │ CategoricalValue{String, UInt32} │
│ race     │ Multiclass{3}              │ CategoricalValue{String, UInt32} │
│ fate     │ Multiclass{2}              │ CategoricalValue{String, UInt32} │
│ apache   │ Union{Missing, Continuous} │ Union{Missing, Float64}          │
│ o2del    │ Union{Missing, Continuous} │ Union{Missing, Float64}          │
│ followup │ Continuous                 │ Float64                          │
│ temp0    │ Continuous                 │ Float64                          │
│ temp10   │ Union{Missing, Continuous} │ Union{Missing, Float64}          │
└──────────┴────────────────────────────┴──────────────────────────────────┘

Let’s take a look at the distribution of baseline temperature.

qq_plot(bernard.temp0; ylab="Baseline Temperature (°C)")
Theoretical quantiles 32.5 35.0 37.5 40.0 42.5 32.5 33.0 33.5 34.0 34.5 35.0 35.5 36.0 36.5 37.0 37.5 38.0 38.5 39.0 39.5 40.0 40.5 41.0 41.5 42.0 42.5 32.45 32.50 32.55 32.60 32.65 32.70 32.75 32.80 32.85 32.90 32.95 33.00 33.05 33.10 33.15 33.20 33.25 33.30 33.35 33.40 33.45 33.50 33.55 33.60 33.65 33.70 33.75 33.80 33.85 33.90 33.95 34.00 34.05 34.10 34.15 34.20 34.25 34.30 34.35 34.40 34.45 34.50 34.55 34.60 34.65 34.70 34.75 34.80 34.85 34.90 34.95 35.00 35.05 35.10 35.15 35.20 35.25 35.30 35.35 35.40 35.45 35.50 35.55 35.60 35.65 35.70 35.75 35.80 35.85 35.90 35.95 36.00 36.05 36.10 36.15 36.20 36.25 36.30 36.35 36.40 36.45 36.50 36.55 36.60 36.65 36.70 36.75 36.80 36.85 36.90 36.95 37.00 37.05 37.10 37.15 37.20 37.25 37.30 37.35 37.40 37.45 37.50 37.55 37.60 37.65 37.70 37.75 37.80 37.85 37.90 37.95 38.00 38.05 38.10 38.15 38.20 38.25 38.30 38.35 38.40 38.45 38.50 38.55 38.60 38.65 38.70 38.75 38.80 38.85 38.90 38.95 39.00 39.05 39.10 39.15 39.20 39.25 39.30 39.35 39.40 39.45 39.50 39.55 39.60 39.65 39.70 39.75 39.80 39.85 39.90 39.95 40.00 40.05 40.10 40.15 40.20 40.25 40.30 40.35 40.40 40.45 40.50 40.55 40.60 40.65 40.70 40.75 40.80 40.85 40.90 40.95 41.00 41.05 41.10 41.15 41.20 41.25 41.30 41.35 41.40 41.45 41.50 41.55 41.60 41.65 41.70 41.75 41.80 41.85 41.90 41.95 42.00 42.05 42.10 42.15 42.20 42.25 42.30 42.35 42.40 42.45 42.50 32.5 35.0 37.5 40.0 42.5 41.457887810060141.52808302808303 41.0695261641197141.25091575091575 40.87426322381859541.05616689456106 40.7395046384847940.95077084272336 40.6352690330556540.64949933658737 40.549628210837840.38888719346788 40.476573287868840.34571366083054 40.4126354132672340.3000005086263 40.3556235879519440.18234417814038 40.30406218599025440.02893728738302 40.2569082559760739.97350467779698 40.2133963737223139.923345908140526 40.1729475552477839.86849727793637 40.1351128405385239.83333163791233 40.0995367958658539.83333163791233 40.0659330322439239.83333163791233 40.0340672744156339.817874391437016 40.0037453412388539.71282111273872 39.97480441714565639.606007931142216 39.9471065866003139.55428495861236 39.92053396029416539.4999991522895 39.8949849435993539.4999991522895 39.8703713395050739.474969009194474 39.8466160710384639.42451724843082 39.8236513702624639.3999989827474 39.80141732332380439.3999989827474 39.7798606904722339.3950913817018 39.75893394076910439.3888897365994 39.73859445611050539.3888897365994 39.7188038700214439.364714281317376 39.69952751464701639.33333502875434 39.68073395529870639.33333502875434 39.662394596376539.319048079233326 39.6444833458797539.2999988132053 39.6269763283202339.2999988132053 39.60985163786429639.29062130919888 39.5930891251002439.27777608235677 39.57667021206139439.25457741314675 39.5605777311120139.22222137451172 39.54479578408168539.213039235172104 39.52930961865846639.18629956740308 39.51410551955477539.16666666666667 39.4991707123691439.144078488751646 39.4844932784006539.106617714459205 39.4700620789463639.09999847412109 39.45586668783839.09999847412109 39.4418973311604839.09999847412109 39.4281448332507139.09999847412109 39.4146005682042439.08251500391698 39.4012564162268539.05555725097656 39.3881047242590839.05555725097656 39.37513827037959439.05555725097656 39.3623502315585939.05555725097656 39.3497341543884439.03431053301353 39.3372839284664638.99999830457899 39.3249937621452638.99999830457899 39.3128581604014938.99999830457899 39.30087190460374638.99999830457899 39.28903003398665638.99999830457899 39.2773278286605438.99999830457899 39.2657607940058338.99999830457899 39.25432464631839538.99999830457899 39.2430152995868938.99999830457899 39.2318288532961438.97997419010297 39.2207615811621338.94444359673394 39.2098099207141238.94444359673394 39.19897046364819538.94444359673394 39.1882399468844938.92881442338992 39.17761524426704538.8999981350369 39.16709335885142538.8999981350369 39.1566714157306338.8999981350369 39.14634665535468438.8999981350369 39.1361164273034138.8999981350369 39.1259781844759138.8999981350369 39.1159294776635338.8962624984201 39.1059679504761738.88888888888889 39.0960913345946938.88888888888889 39.08629744532423438.88888888888889 39.07658417742598538.88888888888889 39.0669495012064438.88888888888889 39.0573914588453338.88888888888889 39.04790816094478638.88888888888889 39.03849778328389438.88888888888889 39.0291585637640638.871306739592875 39.0198887995317938.83333418104384 39.0106868442666438.83333418104384 39.0015511056229538.82300361754403 38.9924800428150938.7999979654948 38.9834721643364338.7999979654948 38.97452602580336438.7999979654948 38.965640227916138.7999979654948 38.95681341452869638.7999979654948 38.9480442708213838.7999979654948 38.9393315215685938.7999979654948 38.9306739294968938.7999979654948 38.92207029372738.7999979654948 38.91351944829495638.79360099298759 38.9050202607474938.77777947319878 38.8965716308072638.77777947319878 38.8881724891036338.762149703662764 38.8798217959653438.72222052680122 38.8715185402712838.71606771355002 38.8632617383561438.70000203450521 38.8550504329676838.70000203450521 38.8468836922728638.70000203450521 38.83876060890993638.70000203450521 38.83068029908401538.70000203450521 38.82264190170375438.70000203450521 38.8146445775568538.70000203450521 38.8066875085222738.70000203450521 38.79876989681732638.691429864792596 38.7908909642775838.66666581895616 38.7830499516680138.66666581895616 38.7752461180237838.66666581895616 38.7674787400189638.66666581895616 38.7597471113619338.66666581895616 38.7520505422160438.66666581895616 38.744388358644238.65323501046615 38.7367599020763338.61111111111111 38.7291645287983938.61111111111111 38.7216016094621138.61111111111111 38.7140705286141738.61111111111111 38.7065706842441838.61111111111111 38.69910148735025538.61111111111111 38.69166236152166538.61111111111111 38.68425274253742438.61111111111111 38.6768720779803738.60864510262551 38.6695198268657938.60000186496311 38.6621954592840638.60000186496311 38.6548984560566138.60000186496311 38.6476283084045838.60000186496311 38.64038451762963438.60000186496311 38.63316659480642638.60000186496311 38.6259740604860338.60000186496311 38.61880644441012638.60000186496311 38.6116632852351738.60000186496311 38.6045441302663538.5911127726237 38.5974485352007838.55555640326606 38.590376063879638.55555640326606 38.5833262880484738.55555640326606 38.5762987871263338.55555640326606 38.5692931479818938.55555640326606 38.5623089647175638.54517805125105 38.5553458384605738.50000169542101 38.5484033771609638.50000169542101 38.5414811953961338.50000169542101 38.5345789141816838.50000169542101 38.52769616078832438.50000169542101 38.52083256856463638.50000169542101 38.5139877767653338.50000169542101 38.5071614303849138.50000169542101 38.5003531799964638.50000169542101 38.4935626815953738.50000169542101 38.48678959644783538.50000169542101 38.4800335909438438.50000169542101 38.4732943364545938.50000169542101 38.4665715091941638.50000169542101 38.4598647900851338.50000169542101 38.4531738646281538.50000169542101 38.44649842277528538.50000169542101 38.4398381588068938.4918204879295 38.43319277121203538.44444274902344 38.4265619625721538.44444274902344 38.4199454394480338.44444274902344 38.41334291226980538.44444274902344 38.4067540952299538.44444274902344 38.40017870617910538.43848469493153 38.39361646652475538.40000152587891 38.3870671011324338.40000152587891 38.38053033822957638.40000152587891 38.37400590931182638.40000152587891 38.3674935490516838.40000152587891 38.3609929952094238.40000152587891 38.3545039885463238.40000152587891 38.3480262727398638.40000152587891 38.3415595943010738.40000152587891 38.3351037024938338.40000152587891 38.3286583492560538.39878026382391 38.3222232891226638.38888804117839 38.3157982791503938.38888804117839 38.3093830788442838.38888804117839 38.3029774500857238.38888804117839 38.2965811570622138.38888804117839 38.2901939661984738.38888804117839 38.2838156460891738.383637816041386 38.2774459674329538.33333333333334 38.2710847029678638.33333333333334 38.2647316274080738.33333333333334 38.2583865173818538.33333333333334 38.25204915137075538.33333333333334 38.2457193096499838.33333333333334 38.239396774229838.33069607801055 38.2330813287981338.30000135633681 38.2267727586640738.30000135633681 38.220470850702538.30000135633681 38.21417539329955638.30000135633681 38.2078861762990838.30000135633681 38.2016029909499138.30000135633681 38.19532562985405438.30000135633681 38.18905388691559538.30000135633681 38.1827875572904738.30000135633681 38.17652643733685438.29873148600261 38.1702703245664338.27777862548828 38.1640190175961338.27777862548828 38.15777231610067638.27777862548828 38.1515300207656838.27777862548828 38.14529193324127538.27521456204928 38.13905785609641438.22222391764323 38.1328275927735938.22129593547593 38.1266009475441138.20000118679471 38.1203777254638338.20000118679471 38.1141577323293338.20000118679471 38.107940774634538.20000118679471 38.10172665952753638.20000118679471 38.09551519476832538.20000118679471 38.08930618868609538.20000118679471 38.0830994501374838.20000118679471 38.0768947884647938.20000118679471 38.0706920134546538.20000118679471 38.06449093529677438.20000118679471 38.0582913645430738.20000118679471 38.05209311206685638.20000118679471 38.0458959890223338.20000118679471 38.0396998068041338.20000118679471 38.0335043770070738.20000118679471 38.0273095113859838.20000118679471 38.021115021815638.19992792038691 38.01492072025061538.16666497124566 38.00872641868562638.16666497124566 38.0025319291152538.11135445991861 37.9963370634941638.11111026340061 37.990141633697138.11111026340061 37.983945451478938.11111026340061 37.9777483284343738.11111026340061 37.9715500759581638.11111026340061 37.96535050520445538.11111026340061 37.9591494270465838.11111026340061 37.9529466520364438.11111026340061 37.9467419903637538.11111026340061 37.94053525181513538.10029400835981 37.93432624573290538.10000101725261 37.92811478097369438.10000101725261 37.9219006658667338.10000101725261 37.915683708171938.10000101725261 37.909463715037438.10000101725261 37.9032404929571238.10000101725261 37.8970138477276438.10000101725261 37.89078358440481538.10000101725261 37.88454950725995538.10000101725261 37.8783114197355538.10000101725261 37.87206912440055438.10000101725261 37.865822422905138.10000101725261 37.859571115934838.057997613890564 37.85331500316437638.05555555555556 37.8470538832107638.05555555555556 37.84078755358563538.05555555555556 37.83451581064717638.05555555555556 37.8282384495513238.05555555555556 37.8219552642021538.05555555555556 37.81566604720167438.05555555555556 37.8093705897987338.0040300902575 37.8030686818371638.0000008477105 37.796760111703138.0000008477105 37.7904446662714338.0000008477105 37.7841221308512538.0000008477105 37.77779228913047438.0000008477105 37.7714549231193838.0000008477105 37.7651098130931638.0000008477105 37.7587567375333738.0000008477105 37.75239547306827638.0000008477105 37.7460257944120638.0000008477105 37.7396474743027638.0000008477105 37.7332602834390238.0000008477105 37.7268639904155138.0000008477105 37.7204583616569538.0000008477105 37.7140431613508438.0000008477105 37.7076181513785738.0000008477105 37.7011830912451838.0000008477105 37.694737738007438.0000008477105 37.6882818462001637.95079524933347 37.6818151677613837.94444613986546 37.6753374519549137.94444613986546 37.6688484452918137.9053732065054 37.6623478914495537.9000006781684 37.655835531189437.9000006781684 37.6493111022716537.9000006781684 37.642774339368837.9000006781684 37.63622497397647537.9000006781684 37.62966273432212537.9000006781684 37.6230873452712837.89040155841608 37.61649852823142537.88888719346788 37.609896001053237.88888719346788 37.6032794779290837.841268872457846 37.59664866928919537.83333248562283 37.5900032816943437.83333248562283 37.58334301772594437.804981990902704 37.5766675758730837.8000005086263 37.569976650416137.8000005086263 37.5632699313070737.8000005086263 37.55654710404663637.8000005086263 37.5498078495573937.8000005086263 37.54305184405339437.8000005086263 37.5362787589058637.8000005086263 37.5294882605047737.8000005086263 37.5226800101163237.8000005086263 37.51585366373589637.8000005086263 37.50900887193659437.78163622994708 37.50214527971290637.77777777777778 37.4952625263195537.77777777777778 37.488360245105137.77777777777778 37.4814380633402737.77777777777778 37.4744956020406637.77777777777778 37.4675324757836737.77777777777778 37.4605482925193437.77777777777778 37.45354265337489637.77777777777778 37.4465151524527637.77777777777778 37.4394653766216337.73308981498373 37.43239290530044637.72222306993272 37.4252973102348837.70444488525391 37.4181781552660637.70000033908421 37.41103499609110437.70000033908421 37.403867380015237.70000033908421 37.39667484569480437.70000033908421 37.38945692287159537.70000033908421 37.3822131320966537.70000033908421 37.3749429844446237.70000033908421 37.3676459812171737.70000033908421 37.3603216136354437.70000033908421 37.3529693625208637.67406732840998 37.34558869796380537.66666836208768 37.33817907897956437.62368649586362 37.33073995315097437.61110941569011 37.3232707562570537.61110941569011 37.3157709118870637.6025882576557 37.3082398310391237.6000001695421 37.3006769117028437.6000001695421 37.293081538424937.56620208218566 37.2854530818570337.55555470784505 37.2777908982851937.55555470784505 37.270094329139337.51367500500801 37.2623627004822737.5 37.2545953224774537.5 37.2467914888332237.5 37.2389504762236537.5 37.231071543683937.5 37.22315393197895637.5 37.2151968629443837.5 37.20719953879747537.5 37.19916114141721437.459219181493964 37.1910808315912937.41191707513271 37.1829577482283737.3999998304579 37.1747910075335537.3999998304579 37.1665797021450937.3999998304579 37.1583229002299537.3919669909355 37.1500196445358937.348839299873845 37.141668951397637.30937656873021 37.1332698096939737.2999996609158 37.1248211797537437.2999996609158 37.11632199220627437.28417512290499 37.10777114677422637.27777693006728 37.0991675110043437.27777693006728 37.0905099189326437.27777693006728 37.0817971696798537.27777693006728 37.0730280259725337.27777693006728 37.0642012125851337.238949683705236 37.05531541469786637.22222222222222 37.04636927616479637.22222222222222 37.0373613976861437.22222222222222 37.0282903348782837.22222222222222 37.0191545962345937.20693493704511 37.0099526409694437.1999994913737 37.0006828767371737.1999994913737 36.99134365721733537.1999994913737 36.9819332795564437.1999994913737 36.972449981655937.1999994913737 36.9628919392947937.1999994913737 36.95325726307524437.177582821129874 36.94354399517699537.16666751437717 36.9337501059065437.16666751437717 36.9238734900250637.12967174190013 36.913911962837737.103736383719905 36.9038632560253237.0999993218316 36.89372501319781637.0999993218316 36.88349478514654637.0999993218316 36.873170024770637.0999993218316 36.86274808164980537.07098766881177 36.85222619623418437.019412775470514 36.8416014936167436.9999991522895 36.83087097685303536.9999991522895 36.8200315197871136.9999991522895 36.809079859339136.9999991522895 36.7980125872050936.9999991522895 36.7868261409143436.9999991522895 36.77551679418283536.9999991522895 36.764080646495436.96483485369747 36.7525136118406936.916409614758614 36.74081140651457436.837141733320934 36.72896953589748436.7999988132053 36.7169832800997436.7999988132053 36.7048476783559736.78617676302918 36.6925575120347736.743344373318735 36.68010728611278636.72222137451172 36.6674912089426436.70854584783571 36.65470317012163536.69999864366319 36.6417367162421536.69999864366319 36.6285850242743836.69999864366319 36.6152408722969936.69999864366319 36.6016966072505236.69999864366319 36.5879441093407536.69999864366319 36.5739747526632336.67999945746528 36.5597793615548736.66666666666667 36.5453481621005836.63357803847764 36.5306707281320936.6045171437246 36.51573592094645436.57372443230597 36.5005318218427636.52283242101052 36.48504565641954436.49999830457899 36.4692637093892236.49999830457899 36.45317122843983536.49999830457899 36.4367523154009936.4032964834509 36.41998980263693436.33333418104384 36.40286511218136.31413838220022 36.3853580946214836.2999979654948 36.3674468441247336.2999979654948 36.3491074852025236.255724654238456 36.3303139258542136.15921782515716 36.3110375704797936.10483621823191 36.29124698439072536.074995187612686 36.27090749973212536.024421347221036 36.24998075002899635.944177424951356 36.22842411717742635.8938219398806 36.2061900702387735.85811927991036 36.1832253694627735.814945803166026 36.1594701009961635.757266499911765 36.1348564969018835.71006246713492 36.1093074802070735.68183111766028 36.0827348539009235.63047596764943 36.05503702335557435.57597116323618 36.0260960992623835.525641482100525 35.995774166085635.5000008477105 35.9639084082573135.44659416375725 35.9303046446353835.33499421246613 35.8947285999627135.24835209735875 35.8568938852534535.21050118992471 35.8164450667789235.14747277750055 35.7729331845251635.1000001695421 35.72577925451097535.047912169143245 35.6742178525492934.8962635603871 35.61720602723434.7999996609158 35.55326815263243434.7999996609158 35.4802132296634334.74879078288655 35.3945724074455834.65467704987206 35.2903368020164434.55470125899356 35.15557821668263534.3989000797854 34.9603152763815234.14900954622634 34.5719536304411333.54901065290775 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 32.5 35.0 37.5 40.0 42.5 33.0 33.5 34.0 34.5 35.0 35.5 36.0 36.5 37.0 37.5 38.0 38.5 39.0 39.5 40.0 40.5 41.0 41.5 42.0 33.10 33.15 33.20 33.25 33.30 33.35 33.40 33.45 33.50 33.55 33.60 33.65 33.70 33.75 33.80 33.85 33.90 33.95 34.00 34.05 34.10 34.15 34.20 34.25 34.30 34.35 34.40 34.45 34.50 34.55 34.60 34.65 34.70 34.75 34.80 34.85 34.90 34.95 35.00 35.05 35.10 35.15 35.20 35.25 35.30 35.35 35.40 35.45 35.50 35.55 35.60 35.65 35.70 35.75 35.80 35.85 35.90 35.95 36.00 36.05 36.10 36.15 36.20 36.25 36.30 36.35 36.40 36.45 36.50 36.55 36.60 36.65 36.70 36.75 36.80 36.85 36.90 36.95 37.00 37.05 37.10 37.15 37.20 37.25 37.30 37.35 37.40 37.45 37.50 37.55 37.60 37.65 37.70 37.75 37.80 37.85 37.90 37.95 38.00 38.05 38.10 38.15 38.20 38.25 38.30 38.35 38.40 38.45 38.50 38.55 38.60 38.65 38.70 38.75 38.80 38.85 38.90 38.95 39.00 39.05 39.10 39.15 39.20 39.25 39.30 39.35 39.40 39.45 39.50 39.55 39.60 39.65 39.70 39.75 39.80 39.85 39.90 39.95 40.00 40.05 40.10 40.15 40.20 40.25 40.30 40.35 40.40 40.45 40.50 40.55 40.60 40.65 40.70 40.75 40.80 40.85 40.90 40.95 41.00 41.05 41.10 41.15 41.20 41.25 41.30 41.35 41.40 41.45 41.50 41.55 32.5 35.0 37.5 40.0 42.5 Baseline Temperature (°C)
Figure 1: Distribution of baseline temperature.

Let’s assume normality and estimate the 95% CI around the mean baseline temperature for all patients.

cis(bernard.temp0)
(outcome = 38.0149207202506,
 lower = 37.91149233237877,
 upper = 38.11834910812243,)

What about the 95% CI around the mean temperature after 36 hr of treatment?

cis(@subset(bernard, !ismissing(:temp10)).temp10)
(outcome = 37.31448060654226,
 lower = 37.21970170268702,
 upper = 37.409259510397504,)
Note

We included the argument !ismissing to remove missing values from the calulation, else the result would be missing.

Tip

We can estimate bootstrap CI via bst from R package bst.

pubh.bst(bernard.temp10) |> rcopy
1×5 DataFrame
Row stat estimate %CI lower upper
String Float64 Float64 Float64 Float64
1 mean 37.31 95.0 37.22 37.4

2.1 Tests for Means

2.1.1 One-sample \(t\)-tests

Perform the following two-sided one-sample \(t\)-test, where the normal core temperature is 37\(^{\circ}C\). Note that because sepsis was diagnosed in this case by a set of symptoms including fever, you would be very surprised if there were no evidence of a difference between the mean baseline temperature of sepsis patients and the normal body temperature.

If we define \(\bar x\) as the mean baseline temperature, our two hypotheses are:

  • \(H_0: \bar x = 37^{\circ}C\)
  • \(H_A: \bar x \neq 37^{\circ}C\)

Take a look at the help file of t.test to get familiarised with its options. By default, we are using a two-sided test, with a significant \(\alpha=0.05\) (95% CI).

OneSampleTTest(bernard.temp0, 37)
One sample t-test
-----------------
Population details:
    parameter of interest:   Mean
    value under h_0:         37
    point estimate:          38.0149
    95% confidence interval: (37.91, 38.12)

Test summary:
    outcome with 95% confidence: reject h_0
    two-sided p-value:           <1e-60

Details:
    number of observations:   455
    t-statistic:              19.23306214688628
    degrees of freedom:       454
    empirical standard error: 0.05276958564889316
Note

We are making a one-sample test, comparing the mean baseline temperature, against the mean reference value of \(\mu\) = 37 \(\degree\) C. Because the test is two-sided, if our mean value is significantly greater or significantly less than \(\mu\) = 37 \(\degree\) C we reject the null hypothesis. The probability of observing a mean baseline temperature of \(\bar x\) = 37 \(\degree\) C in our sample is \(p\) < 0.001. The mean baseline temperature in our sample was \(\bar x\) = 38.09 \(\degree\) C (95% CI: 37.93 \(\degree\) C, 38.25 \(\degree\) C).

Important

There are deviations from normality in baseline temperature. Lower temperatures are particularly very unlikely to come from a normal distribution.

Our sample is large enough to not be worried about small deviations from normality. In healthy subjects, the temperature would be expected to be centred, and normally distributed.

2.2 Paired \(t\)-tests

Example

Assume we want to know if there was a significant decrease in the mean temperature at 36 hours in the Placebo group. The \(t\)-test assumes that data is independent. In this example, the same subjects were measured twice: at baseline and 36 hours. This is a classic example of a paired analysis.

placebo = @select(
  @subset(bernard, :treat == "Placebo"),
  :temp0, :temp10
)
dropmissing!(placebo);
OneSampleTTest(placebo.temp10, placebo.temp0)
One sample t-test
-----------------
Population details:
    parameter of interest:   Mean
    value under h_0:         0
    point estimate:          -0.498563
    95% confidence interval: (-0.6428, -0.3543)

Test summary:
    outcome with 95% confidence: reject h_0
    two-sided p-value:           <1e-09

Details:
    number of observations:   201
    t-statistic:              -6.816054633057416
    degrees of freedom:       200
    empirical standard error: 0.07314536404958778
Interpretation

The mean decrease in temperature from baseline to 36 hr in the placebo group was 0.45 \(\degree\) C (95% CI: 0.18 \(\degree\) C, 0.71 \(\degree\) C). There was a significant placebo effect (\(p=0.001\)) as the 95% CI for the temperature change in the placebo group did not include the null value of zero.

2.3 Two-sample \(t\)-tests

Our real question of interest is to test if given Ibuprofen was statistically different from given placebo in patients with sepsis. This is a two-sided, two-sample hypothesis. The two samples are independent (treatment groups), and our variable of interest is temp_change.

First, we calculate the difference in temperatures.

bern = @select(bernard, :temp0, :temp10, :treat)
@transform!(bern, :temp_change = :temp10 - :temp0)
dropmissing!(bern);

One of the assumptions is that the distribution of temp_change is normal for each group. The another big assumption is that the variance is the same. To compare variances, we perform a variance test. The null hypothesis is that the ratio of the two variances is equal to one (same variance) and the alternative is that is different from one. A \(p \leq 0.05\) means that there is no statistical difference between the two variances and, therefore, that the assumption of homogeneity of variances holds.

First, we perform a standard descriptive analysis on temp_change.

summarize_by(bern, :treat, :temp_change)
          |             | Obs |  Mean  | Std. Dev. |  Min   |  Max  
--------------------------------------------------------------------
  Placebo | temp_change | 201 | -0.499 |     1.037 | -3.278 | 2.900 
--------------------------------------------------------------------
Ibuprofen | temp_change | 198 | -1.034 |     1.133 | -4.100 | 2.500 
Note

The R package pubh has function estat that allow us to also generate a table of descriptive statistics.

pubh.estat(@formula(temp_change ~ treat), data=bern) |> rcopy
2×9 DataFrame
Row treat N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 temp_change Placebo 201.0 -3.28 2.9 -0.5 -0.5 1.04 -2.08
2 Ibuprofen 198.0 -4.1 2.5 -1.03 -1.11 1.13 -1.1
Exercise

Construct a QQ-plot of temp_change from subjects by treatment group, against the standard normal distribution to check for the normality assumption.

Code
qq_plot(bern.temp_change, ylab = "Temperature Change (°C)")
Theoretical quantiles -5 -4 -3 -2 -1 0 1 2 3 -5.0 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 -5.00 -4.95 -4.90 -4.85 -4.80 -4.75 -4.70 -4.65 -4.60 -4.55 -4.50 -4.45 -4.40 -4.35 -4.30 -4.25 -4.20 -4.15 -4.10 -4.05 -4.00 -3.95 -3.90 -3.85 -3.80 -3.75 -3.70 -3.65 -3.60 -3.55 -3.50 -3.45 -3.40 -3.35 -3.30 -3.25 -3.20 -3.15 -3.10 -3.05 -3.00 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -2.60 -2.55 -2.50 -2.45 -2.40 -2.35 -2.30 -2.25 -2.20 -2.15 -2.10 -2.05 -2.00 -1.95 -1.90 -1.85 -1.80 -1.75 -1.70 -1.65 -1.60 -1.55 -1.50 -1.45 -1.40 -1.35 -1.30 -1.25 -1.20 -1.15 -1.10 -1.05 -1.00 -0.95 -0.90 -0.85 -0.80 -0.75 -0.70 -0.65 -0.60 -0.55 -0.50 -0.45 -0.40 -0.35 -0.30 -0.25 -0.20 -0.15 -0.10 -0.05 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 -5 0 5 2.6081448338598632.8999985642036057 2.2180611528661232.7015019658215325 2.02155589746049062.3354223065972555 1.8857650204137172.1339182268109242 1.78062145882109672.051127058521552 1.6941568126133491.9999991522894973 1.62033956750121361.973126824935874 1.55568666702816841.9230591942094777 1.49799751612282251.9000011923186295 1.44578965374247931.8999989827473949 1.39801506015068531.8526304813853456 1.35390443226555851.7057650615763267 1.3128757407845681.6000007261840872 1.2744775839352861.5533818537330495 1.23835253524205811.453634276425927 1.20421255471925571.3692581444455905 1.17182198134271421.3078542022577626 1.14098545755594751.2270979853401358 1.1115391590768461.141466625684222 1.08334429774708131.1111111111111078 1.05628222335759041.0612650323929895 1.03025067304732951.0000016954210094 1.00516085915066490.9260661062242155 0.98093517953298610.8333333333333357 0.95750539680868930.8187140451769672 0.93481117540003370.7999995079454895 0.91279889497378860.7999971177842906 0.89142067968322090.7999971177842835 0.87063359761824310.7904749068002799 0.85039899574835330.7541068939404485 0.83068194365019550.7128088389954175 0.81145076527195940.700001186794705 0.79267664247166270.7000011867947009 0.77433327747458480.6630189981144478 0.75639660400949320.6065162142416589 0.73884453890612710.600001017252603 0.72165676751436330.6000010172525947 0.7048145575457760.5999992963365424 0.68830059692030290.5820643553259871 0.67209885198333640.5555555555555614 0.6561944430864390.5555555555555542 0.64057353503147580.55555555555555 0.62522324028921640.55555555555555 0.61013153323894960.5338349179018503 0.59528717395093020.5000008477105098 0.58067964026026650.5000008477105043 0.56629906706875380.5000008477104956 0.55213619196732620.49999923302382376 0.53818230640236610.4789737498517874 0.52442921171846390.44444613986545267 0.51086917950244310.4278487118131647 0.497494915731216070.40000067816839646 0.484299528292137850.3959077980086062 0.47127649750062560.3888898810713538 0.458419649287796550.3686981392384773 0.44572313077191030.32130289993987865 0.4331813879626880.300000508626309 0.4207891453779120.30000050862630445 0.40854138737799660.30000050862630195 0.396433341046890540.30000050862629235 0.38446046046744240.299999032037816 0.37261841225649750.29231167875335823 0.36090306224001580.27777777777778323 0.34931046316156910.2777777777777761 0.3378368433291010.2665000412000003 0.32647859611488770.23698110429067537 0.31523227023256120.21481266728154505 0.304094560722855340.20000033908421008 0.29306230058669680.20000033908420534 0.28213245301036350.20000033908420303 0.271302104132908450.18922383524321945 0.260568456310852350.16666836208767932 0.249928821839480840.1525215609961281 0.239380617093901020.11871471739651618 0.228921357056462220.11111365424262497 0.218548650200194650.11111233699822728 0.208260193700693370.1111094156901089 0.19805376895132010.11110941569010456 0.187927237358834320.11110941569009469 0.177878536398547120.10776828902904552 0.16790567590990180.10000016954210622 0.158006734615003030.10000016954210413 0.14817985684408840.10000016954209912 0.138423249453260570.10000016954209498 0.12873517892101050.0871912957034432 0.119113968611140870.055557735382574185 0.109557996190706270.039821169031791843 0.10006569119247210.0 0.090635532712232350.0 0.081266047232055770.0 0.071955806561232570.0 0.062703425887297490.0 0.053507561930093320.0 0.0443669111923404860.0 0.035280208300681570.0 0.026246224431577647-0.014480425102471196 0.01726376581686495-0.05555470784505445 0.008331672324123574-0.06691670557211737 -0.0005511838926249846-0.10000016954210092 -0.009385899676090936-0.100000169542108 -0.01817354208663824-0.10275659903747743 -0.02691514956460539-0.11110941569009469 -0.03561173303739007-0.11110941569009815 -0.04426427697447677-0.11111043549221397 -0.05287374039334114-0.11111365424261149 -0.06144105781900211-0.12420073222131187 -0.06996714019978834-0.1666651114684434 -0.07845287578174198-0.16666836208766966 -0.08689913094390866-0.16666836208766966 -0.0953067509966421-0.1741868531395187 -0.10367656094489319-0.2000003390842014 -0.11200936621836055-0.20000033908421244 -0.12030595337023531-0.204845896938401 -0.12856709074619332-0.22222306993273122 -0.1367935291251663-0.23405803275686254 -0.14498600233335257-0.27777777777776735 -0.15314522783282047-0.27777777777777857 -0.16127190728599805-0.27777777777777857 -0.16936672709725242-0.2822891592282298 -0.17743035893270254-0.3000005086262988 -0.18546346021933546-0.30000050862630445 -0.193466674624445-0.3065165342045717 -0.20144063251634536-0.3333324856228259 -0.2093859514072648-0.3333324856228259 -0.21730323637927296-0.3333324856228259 -0.2251930804940494-0.33333248562282725 -0.2330560651872522-0.3333324856228343 -0.24089276064821652-0.33333325047442147 -0.24870372618565473-0.33333672417535354 -0.25648951058002034-0.3430824205424638 -0.2642506524231311-0.38888719346788037 -0.27198768044565086-0.38888719346788037 -0.279701113832964-0.38888719346788037 -0.287391462529981-0.38888789458182876 -0.2950592275353575-0.38889143202038784 -0.30270490118561066-0.38889143202039494 -0.3103289674295694-0.39064485427232 -0.3179319020935943-0.39999709823808305 -0.3255141731379615-0.40000067816838936 -0.3330762409048083-0.4000006781683904 -0.34061855835799054-0.4000006781683975 -0.34814157131521856-0.40000067816840357 -0.35564571867278705-0.40000067816840357 -0.3631314326232307-0.40000067816840357 -0.3705991388661926-0.40000067816840457 -0.3780492568128075-0.4000006781684107 -0.3854821997838624-0.4059044613010986 -0.3928983752020067-0.4444461398654492 -0.40029818477825263-0.44444613986545534 -0.407682024693016-0.44444613986545534 -0.4150502857719131-0.4512681273224318 -0.422403353656547-0.49999711905904265 -0.42974160897047725-0.5000008477104956 -0.43706542748058697-0.5000008477104956 -0.44437518025402667-0.5000008477104956 -0.4516712338109309-0.5000008477104956 -0.4589539502730702-0.5000008477104964 -0.46622368750862064-0.5000008477105027 -0.4734807992731999-0.5000008477105027 -0.4807256353473383-0.5000008477105027 -0.4879585416705203-0.5000008477105033 -0.49517986047195334-0.5000008477105098 -0.5023899303981894-0.5000008477105098 -0.5095890866377429-0.5000008477105098 -0.5167776610428231-0.5000008477105098 -0.5239559822483113-0.5000008477105098 -0.5311243757880946-0.5045955979834058 -0.5382831642088773-0.5555555555555506 -0.5454326671815721-0.5555555555555571 -0.5525732016103858-0.5555555555555577 -0.559705081739693-0.5587856193931341 -0.5668286192588061-0.5999967787000823 -0.5739441234047293-0.5999970655194375 -0.581051901062996-0.6000010172525951 -0.5881522568666728-0.6000010172526018 -0.5952454932936269-0.6000010172526022 -0.6023319107621273-0.6006413998626432 -0.6094118077248785-0.6111102634005978 -0.6164854807615501-0.614034195392448 -0.6235532246698935-0.6666649712456589 -0.6306153325555103-0.6666651730814925 -0.6376720959203555-0.6681727167956395 -0.6447238047500365-0.699996948242181 -0.6517707475999897-0.6999971182092121 -0.6588132116805913-0.700001186794694 -0.6658514829412792-0.7000011867947011 -0.672885846153743-0.700001186794708 -0.6799165849942541-0.700001186794708 -0.6869439821251896-0.700001186794708 -0.6939683192758225-0.7000011867947082 -0.700989877322428-0.7000011867947151 -0.7080089363677758-0.7004467553330311 -0.7150257758200573-0.7231984872799433 -0.7220406744713164-0.7777744506733926 -0.729053910575431-0.7777786254882743 -0.7360657619257115-0.7780013672656846 -0.743076505932164-0.7999971496530954 -0.7500864196984823-0.8000013563367929 -0.7570957800988162-0.8000013563368 -0.7641048638543788-0.8000013563368 -0.7711139476099413-0.8000013563368071 -0.7781233080102753-0.8000013563368071 -0.7851332217765935-0.8000013563368142 -0.7921439657830461-0.8000013563368142 -0.7991558171333265-0.8329156393860043 -0.8061690532374413-0.8333333333333286 -0.8131839518887002-0.8333333333333426 -0.8202007913409818-0.8877741623243546 -0.8272198503863294-0.8888880411783902 -0.8342414084329351-0.8888880411783902 -0.8412657455835679-0.8888921628785786 -0.8482931427145035-0.8996633021355375 -0.8553238815550144-0.9000013877807036 -0.8623582447674785-0.9000015258789131 -0.8693965160281663-0.9000015258789134 -0.876438980108768-0.9426606448371835 -0.883485922958721-0.9444427490234372 -0.8905376317884022-0.9444427490234375 -0.8975943951532472-0.9973519945901431 -0.9046565030388641-0.9999974568684777 -0.9117242469472074-0.9999974568684845 -0.9187979199838792-0.9999974568685049 -0.9258778169466302-1.000001451093411 -0.9329642344151308-1.0000016954209983 -0.9400574708420847-1.0000016954209983 -0.9471578266457616-1.000001695421005 -0.9542656043040281-1.0000016954210054 -0.9613811084499515-1.0000016954210054 -0.9685046459690645-1.0000016954210054 -0.9756365260983718-1.000001695421012 -0.9827770605271853-1.0000016954210125 -0.9899265634998803-1.0000016954210125 -0.9970853519206628-1.0000016954210125 -1.0042537454604463-1.0000016954210125 -1.0114320666659344-1.0000016954210125 -1.0186206410710148-1.0000016954210125 -1.025819797310568-1.0504008680623598 -1.0330298672368043-1.0555559995943917 -1.0402511860382373-1.095653747456608 -1.0474840923614193-1.0999976264105904 -1.0547289284355577-1.099997626410591 -1.061986040200137-1.1000014187996756 -1.0692557774356874-1.1000018649631038 -1.0765384938978269-1.1000018649631045 -1.0838345474547308-1.1000018649631107 -1.0911443002281707-1.1000018649631116 -1.09846811873828-1.1000018649631116 -1.1058063740522106-1.1000018649631116 -1.1131594419368445-1.1097468177245176 -1.1205277030157417-1.1111111111111143 -1.1279115429305049-1.1111111111111143 -1.1353113525067509-1.1111111111111143 -1.1427275279248952-1.1111111111111143 -1.15016047089595-1.1591471366914126 -1.157610588842565-1.1666658189561607 -1.165078295085527-1.1666658189561738 -1.1725640090359706-1.1666694520011813 -1.1800681563935391-1.1951531622941631 -1.1875911693507668-1.1999977959526902 -1.1951334868039494-1.1999977959526902 -1.2026955545707958-1.2000013865059533 -1.2102778256151634-1.2000020345052107 -1.2178807602791881-1.2000020345052227 -1.225504826523147-1.2186566583627554 -1.2331505001734-1.222220526801209 -1.2408182651787767-1.222220526801209 -1.2485086138757935-1.22222405361684 -1.2562220472631067-1.2222247653537295 -1.2639590752856262-1.2681687625129734 -1.2717202171287374-1.2777752346462705 -1.2795060015231026-1.2777752346462705 -1.287316967060541-1.2777752346462705 -1.2951536625215052-1.2777752346462705 -1.3030166472147082-1.2958799089704252 -1.3109064913294843-1.3000022040473098 -1.318823776301493-1.3000022040473098 -1.3267690951924123-1.3000022040473098 -1.3347430530843125-1.3000022040473098 -1.342746267489422-1.3000022040473098 -1.3507793687760552-1.3000022040473098 -1.3588430006115049-1.3265641610026846 -1.3669378204227596-1.3333333099628635 -1.3750644998759372-1.3333341810438368 -1.3832237253754052-1.3333341810438384 -1.391416198583591-1.3770539260647474 -1.3996426369625643-1.3888888888888786 -1.407903774338522-1.3888888888888953 -1.416200361490397-1.3975479754854523 -1.4245331667638643-1.399998135036894 -1.4329029767121155-1.4000014175249322 -1.4413105967648487-1.400002373589409 -1.4497568519270156-1.4000023735894145 -1.458242587508969-1.4340851161513817 -1.4667686698897555-1.4444435967339402 -1.4753359873154164-1.4444435967339402 -1.4839454507342809-1.4444435967339402 -1.4925979946713674-1.4444435967339402 -1.5012945781441522-1.4695919698674225 -1.5100361856221194-1.4944854455882477 -1.5188238280326667-1.4999983045789929 -1.5276585438161328-1.4999983045789946 -1.5365414000328812-1.5000014595917572 -1.5454734935256225-1.5000025431315043 -1.5544559521403354-1.500002543131508 -1.5634899360094392-1.500002543131508 -1.572576638901098-1.5000025431315132 -1.581717289638851-1.5000025431315152 -1.590913153596055-1.5000025431315152 -1.60016553426999-1.5000025431315152 -1.6094757749408135-1.5000025431315152 -1.61884526042099-1.5400991224704867 -1.6282754189012296-1.5875225550482344 -1.637767723899464-1.5999984741210866 -1.6473236963198985-1.5999984741210918 -1.6569449066297681-1.5999984741210989 -1.6666329771620183-1.5999984741211009 -1.676389584552846-1.6000014697897846 -1.6862164623237605-1.60000271267361 -1.6961154036186594-1.6000027126736143 -1.7060882641073047-1.6000027126736143 -1.716136965067592-1.6000027126736143 -1.7262634966600778-1.6000027126736192 -1.736469921409451-1.6076843720629677 -1.7467583779089524-1.6111106415772272 -1.75713108476522-1.6187437753828808 -1.7675903448026586-1.6526325961999677 -1.7781385495482385-1.666666666666662 -1.78877818401961-1.6666666666666692 -1.799511831841666-1.6892221398222076 -1.8103421807191211-1.6999986436631953 -1.8212720282954544-1.7000014906107577 -1.832304288431613-1.7000028822157205 -1.8434419979413188-1.7000028822157205 -1.8546883238236453-1.7000028822157205 -1.8660465710378586-1.714703839524351 -1.8775201908703267-1.7222213745117188 -1.8891127899487734-1.7222241577216444 -1.9008281399652551-1.758563138090178 -1.9126701881762-1.7777788443208191 -1.9246430687556482-1.7922028509961652 -1.9367511150867542-1.799998813205294 -1.9489988730866696-1.8214682820977952 -1.9613911156714456-1.8333335096741143 -1.973932858480668-1.8688399172267307 -1.9866293769965542-1.8888897365993833 -1.9994862252093832-1.8888897365993884 -2.0125092560008957-1.8888897365993973 -2.0257046434399735-1.8888897365994026 -2.0390789072112008-1.895850417143264 -2.0526389394272213-1.9000016278591287 -2.0663920341111237-1.9276258361867478 -2.080345919676084-1.9788354540628084 -2.0945087947775116-1.9999991522894902 -2.108889367969024-1.9999991522894947 -2.1234969016596876-1.9999991522894973 -2.1383412609477075-2.0000017336635842 -2.153432967997974-2.033695655425448 -2.1687832627402335-2.055553860134542 -2.1844041707951964-2.055553860134549 -2.200308579692094-2.0555563990168846 -2.2165103246290605-2.0820668984574873 -2.2330242852545337-2.099999321831605 -2.249866495223121-2.1065727112785284 -2.2670542666148847-2.1111128065321196 -2.284606331718251-2.1632418397474797 -2.3025430051833427-2.1999994913736995 -2.3208863701804203-2.2452234657519208 -2.339660492980717-2.290642721611155 -2.358891671358953-2.2999996609157987 -2.378608723457111-2.2999996609158098 -2.3988433253270007-2.3000020829458117 -2.4196304073919785-2.3189647932697945 -2.4410086226825465-2.333334038696461 -2.4630209031087915-2.3646613290734155 -2.485715124517447-2.4200749536703867 -2.509144907241744-2.4444434225177436 -2.5333705868594225-2.4444452921549495 -2.558460400756087-2.5059870988404516 -2.584491951066348-2.5800609774524506 -2.611554025455839-2.606100052122086 -2.6397488867856036-2.6596759933178533 -2.669195185264705-2.699996100531692 -2.700031709051472-2.712028745620976 -2.732422282428013-2.752158564134942 -2.7665622629508158-2.7896967361777296 -2.8026873116440436-2.8177942707823354 -2.8410854684933255-2.8923703169497337 -2.882114159974317-2.973822453200601 -2.926224787859443-2.9999966091579897 -2.9739993814512373-2.999998829352166 -3.02620724383158-3.0868856078965594 -3.083896394736928-3.183959540269127 -3.14854929520997-3.2114703381038288 -3.2223665403221076-3.2507628247253866 -3.3088311865298548-3.306180464793164 -3.413974748122477-3.44639203701799 -3.5497656251692478-3.668058220638393 -3.746270880574883-3.805765679406443 -4.136354561568615-3.9670026179509943 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -5 -4 -3 -2 -1 0 1 2 3 -4.5 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 -4.15 -4.10 -4.05 -4.00 -3.95 -3.90 -3.85 -3.80 -3.75 -3.70 -3.65 -3.60 -3.55 -3.50 -3.45 -3.40 -3.35 -3.30 -3.25 -3.20 -3.15 -3.10 -3.05 -3.00 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -2.60 -2.55 -2.50 -2.45 -2.40 -2.35 -2.30 -2.25 -2.20 -2.15 -2.10 -2.05 -2.00 -1.95 -1.90 -1.85 -1.80 -1.75 -1.70 -1.65 -1.60 -1.55 -1.50 -1.45 -1.40 -1.35 -1.30 -1.25 -1.20 -1.15 -1.10 -1.05 -1.00 -0.95 -0.90 -0.85 -0.80 -0.75 -0.70 -0.65 -0.60 -0.55 -0.50 -0.45 -0.40 -0.35 -0.30 -0.25 -0.20 -0.15 -0.10 -0.05 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 -5 0 5 Temperature Change (°C)
Figure 2: QQ-plot of temperature change against the standard normal distribution.

We perform a variance test with VarianceFTest.

VarianceFTest(
  @subset(bern, :treat == "Ibuprofen").temp_change,
  @subset(bern, :treat == "Placebo").temp_change
) |> pvalue |> r3
0.212
Note

HypothesisTests is not designed to work with data frames, hence, one need to provides each vector. As an alternative to the previous code, we can generate these vectors with the function vec_group. The arguments of vec_group are the data frame, the outcome (continuous) variable and the group (categorical) variable.

VarianceFTest(
  vec_group(bern, :temp_change, :treat)...
) |> pvalue |> r3
0.212

Now, let’s test the null hypothesis that the mean temperature change between the two groups is the same.

EqualVarianceTTest(
  vec_group(bern, :temp_change, :treat)...
)
Two sample t-test (equal variance)
----------------------------------
Population details:
    parameter of interest:   Mean difference
    value under h_0:         0
    point estimate:          0.535107
    95% confidence interval: (0.3214, 0.7489)

Test summary:
    outcome with 95% confidence: reject h_0
    two-sided p-value:           <1e-05

Details:
    number of observations:   [201,198]
    t-statistic:              4.9216804918824
    degrees of freedom:       397
    empirical standard error: 0.10872455012044961

3 Non Parametric tests

3.1 Mann-Whitney

In some disciplines, researchers are not interested in the magnitude of the difference, e.g., when there is no precise knowledge of the interpretation of the scales. Under those circumstances, they may choose a priori, to use non-parametric tests for relatively small samples.

Non-parametric tests are also used when violations to the \(t\)-test assumptions occur.

Important

We never, ever perform both a parametric and a non-parametric test. That decision has to be taken a priori, given our assumptions. When we perform both tests, we may fall into the temptation to report the more beneficial results; in other words, by performing both tests, we introduce bias in our analysis.

Example

We will compare energy expenditure between lean and obese woman.

energy = rcopy(R"ISwR::energy")
energy |> schema
┌─────────┬───────────────┬──────────────────────────────────┐
│ names   │ scitypes      │ types                            │
├─────────┼───────────────┼──────────────────────────────────┤
│ expend  │ Continuous    │ Float64                          │
│ stature │ Multiclass{2} │ CategoricalValue{String, UInt32} │
└─────────┴───────────────┴──────────────────────────────────┘
Exercise

Calculate descriptive statistics for variable expend by stature from the energy dataset.

Code
pubh.estat(@formula(expend ~ stature), data=energy) |> rcopy
2×9 DataFrame
Row stature N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 expend lean 13.0 6.13 10.88 8.07 7.9 1.24 0.15
2 obese 9.0 8.79 12.79 10.3 9.69 1.4 0.14
Question

What are your general observations from the descriptive analysis?

Answer

On average, obese women have more energy expenditure than lean woman, but we do not know if that difference is significant. Both groups of women show the same variance on energy expenditure and show a positive skew on the distribution (the medians are relatively lower than the means).

Given that our samples are relatively small (less than 30 observations per group), the best way to graphically compare distributions is by strip charts.

Exercise

Construct a strip chart comparing the energy expenditure by stature.

Code
strip_error(
  energy,
  :stature, :expend, 
  xlab = "Stature",
  ylab = "Energy expenditure (MJ)"
)
Stature obese lean 110.297777777777778 28.066153846153847 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 6 8 10 12 14 6.0 6.5 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 12.5 13.0 13.5 14.0 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00 4.05 4.10 4.15 4.20 4.25 4.30 4.35 4.40 4.45 4.50 4.55 4.60 4.65 4.70 4.75 4.80 4.85 4.90 4.95 5.00 5.05 5.10 5.15 5.20 5.25 5.30 5.35 5.40 5.45 5.50 5.55 5.60 5.65 5.70 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 6.25 6.30 6.35 6.40 6.45 6.50 6.55 6.60 6.65 6.70 6.75 6.80 6.85 6.90 6.95 7.00 7.05 7.10 7.15 7.20 7.25 7.30 7.35 7.40 7.45 7.50 7.55 7.60 7.65 7.70 7.75 7.80 7.85 7.90 7.95 8.00 8.05 8.10 8.15 8.20 8.25 8.30 8.35 8.40 8.45 8.50 8.55 8.60 8.65 8.70 8.75 8.80 8.85 8.90 8.95 9.00 9.05 9.10 9.15 9.20 9.25 9.30 9.35 9.40 9.45 9.50 9.55 9.60 9.65 9.70 9.75 9.80 9.85 9.90 9.95 10.00 10.05 10.10 10.15 10.20 10.25 10.30 10.35 10.40 10.45 10.50 10.55 10.60 10.65 10.70 10.75 10.80 10.85 10.90 10.95 11.00 11.05 11.10 11.15 11.20 11.25 11.30 11.35 11.40 11.45 11.50 11.55 11.60 11.65 11.70 11.75 11.80 11.85 11.90 11.95 12.00 12.05 12.10 12.15 12.20 12.25 12.30 12.35 12.40 12.45 12.50 12.55 12.60 12.65 12.70 12.75 12.80 12.85 12.90 12.95 13.00 13.05 13.10 13.15 13.20 13.25 13.30 13.35 13.40 13.45 13.50 13.55 13.60 13.65 13.70 13.75 13.80 13.85 13.90 13.95 14.00 5 10 15 Energy expenditure (MJ)
Figure 3: Comparison of energy expenditure between obese and lean women.

We can check graphically for normality. Strictly speaking, the mean difference is the one that has to be normally distributed, for simplicity, we will look at the distribution of energy for each group, as that is a good indicator about normality on the mean difference.

let
  energy_qq = DataFrames.combine(
    groupby(energy, :stature),
    :expend => (x->fit(Normal, x)) => :d
  )
  
  plot(
    energy_qq,
    x=:d, y=energy.expend, color=:stature,
    Stat.qq,
    Guide.xlabel("Normal quantiles"),
    Guide.ylabel("Energy expenditure (MJ)")
  )
end
Normal quantiles 5 6 7 8 9 10 11 12 13 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 12.5 13.0 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00 4.05 4.10 4.15 4.20 4.25 4.30 4.35 4.40 4.45 4.50 4.55 4.60 4.65 4.70 4.75 4.80 4.85 4.90 4.95 5.00 5.05 5.10 5.15 5.20 5.25 5.30 5.35 5.40 5.45 5.50 5.55 5.60 5.65 5.70 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 6.25 6.30 6.35 6.40 6.45 6.50 6.55 6.60 6.65 6.70 6.75 6.80 6.85 6.90 6.95 7.00 7.05 7.10 7.15 7.20 7.25 7.30 7.35 7.40 7.45 7.50 7.55 7.60 7.65 7.70 7.75 7.80 7.85 7.90 7.95 8.00 8.05 8.10 8.15 8.20 8.25 8.30 8.35 8.40 8.45 8.50 8.55 8.60 8.65 8.70 8.75 8.80 8.85 8.90 8.95 9.00 9.05 9.10 9.15 9.20 9.25 9.30 9.35 9.40 9.45 9.50 9.55 9.60 9.65 9.70 9.75 9.80 9.85 9.90 9.95 10.00 10.05 10.10 10.15 10.20 10.25 10.30 10.35 10.40 10.45 10.50 10.55 10.60 10.65 10.70 10.75 10.80 10.85 10.90 10.95 11.00 11.05 11.10 11.15 11.20 11.25 11.30 11.35 11.40 11.45 11.50 11.55 11.60 11.65 11.70 11.75 11.80 11.85 11.90 11.95 12.00 12.05 12.10 12.15 12.20 12.25 12.30 12.35 12.40 12.45 12.50 12.55 12.60 12.65 12.70 12.75 12.80 12.85 12.90 12.95 13.00 5 10 15 lean obese stature 12.93418759582587912.341363636363637 12.2607887640144911.703181818181818 11.88905989703298611.266590909090908 11.61333284886060810.631136363636365 11.38571825910448610.096818181818183 11.1867052164724139.9 11.0061880719110169.687954545454545 10.8381053726649639.605227272727273 10.6784070760032259.207727272727274 10.5241276970320719.162727272727274 10.3728991187901348.781136363636364 10.2226564367654238.408863636363636 10.0714278585234858.129772727272726 9.9171484795523318.092272727272727 9.7574501828905948.08159090909091 9.589367483644547.936818181818182 9.4088503390831437.66 9.209837296451077.544772727272727 8.9822227066949487.497045454545455 8.706495658522577.48 8.3347667915410647.235681818181819 7.6613679597296766.569090909090908 10.4456754330463212.341363636363637 9.83789167882342511.703181818181818 9.50238357329359211.266590909090908 9.25352299830830410.631136363636365 9.04808686842112310.096818181818183 8.868465389326969.9 8.7055375928411879.687954545454545 8.5538326460225449.605227272727273 8.4096951368632499.207727272727274 8.2704485333340529.162727272727274 8.1339554646091878.781136363636364 7.9983522276985088.408863636363636 7.8618591589736428.129772727272726 7.7226125554444458.092272727272727 7.5784750462851518.08159090909091 7.4267700994665077.936818181818182 7.2638423029807347.66 7.0842208238865727.544772727272727 6.8787846939993927.497045454545455 6.62992411901410257.48 6.2944160134842687.235681818181819 5.68663225926137456.569090909090908 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 6 8 10 12 14 6.0 6.5 7.0 7.5 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 12.5 13.0 13.5 14.0 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00 4.05 4.10 4.15 4.20 4.25 4.30 4.35 4.40 4.45 4.50 4.55 4.60 4.65 4.70 4.75 4.80 4.85 4.90 4.95 5.00 5.05 5.10 5.15 5.20 5.25 5.30 5.35 5.40 5.45 5.50 5.55 5.60 5.65 5.70 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 6.25 6.30 6.35 6.40 6.45 6.50 6.55 6.60 6.65 6.70 6.75 6.80 6.85 6.90 6.95 7.00 7.05 7.10 7.15 7.20 7.25 7.30 7.35 7.40 7.45 7.50 7.55 7.60 7.65 7.70 7.75 7.80 7.85 7.90 7.95 8.00 8.05 8.10 8.15 8.20 8.25 8.30 8.35 8.40 8.45 8.50 8.55 8.60 8.65 8.70 8.75 8.80 8.85 8.90 8.95 9.00 9.05 9.10 9.15 9.20 9.25 9.30 9.35 9.40 9.45 9.50 9.55 9.60 9.65 9.70 9.75 9.80 9.85 9.90 9.95 10.00 10.05 10.10 10.15 10.20 10.25 10.30 10.35 10.40 10.45 10.50 10.55 10.60 10.65 10.70 10.75 10.80 10.85 10.90 10.95 11.00 11.05 11.10 11.15 11.20 11.25 11.30 11.35 11.40 11.45 11.50 11.55 11.60 11.65 11.70 11.75 11.80 11.85 11.90 11.95 12.00 12.05 12.10 12.15 12.20 12.25 12.30 12.35 12.40 12.45 12.50 12.55 12.60 12.65 12.70 12.75 12.80 12.85 12.90 12.95 13.00 13.05 13.10 13.15 13.20 13.25 13.30 13.35 13.40 13.45 13.50 13.55 13.60 13.65 13.70 13.75 13.80 13.85 13.90 13.95 14.00 5 10 15 Energy expenditure (MJ)
Figure 4: QQ-plots of 24 hr energy expenditure of women against the standard Normal distribution by stature status.

What about variance equality?

VarianceFTest(
  vec_group(energy, :expend, :stature)...
) |> pvalue |> r3
0.68

The associated non-parametric test to the t-test is the Wilcoxon-Mann-Whitney test, more commonly known as Mann-Whitney test.

MannWhitneyUTest(
  vec_group(energy, :expend, :stature)...
) |> pvalue |> r3
0.002

3.1.1 Paired data

Paired tests are used when there are two measurements on the same experimental unit. We will use data on pre- and post-menstrual energy intake in a group of 11 women.

intake = rcopy(R"ISwR::intake")
intake |> schema
┌───────┬────────────┬─────────┐
│ names │ scitypes   │ types   │
├───────┼────────────┼─────────┤
│ pre   │ Continuous │ Float64 │
│ post  │ Continuous │ Float64 │
└───────┴────────────┴─────────┘

We can start, as usual, with descriptive statistics.

estat(intake)
2×6 DataFrame
Row Variable n Median Mean SD CV
Symbol Int64 Float64 Float64 Float64 Float64
1 pre 11 6515.0 6753.64 1142.12 0.169
2 post 11 5265.0 5433.18 1216.83 0.224

Let’s work on the assumption that we are not interested in the magnitude of the difference but only if that difference is significant or not. On those circumstances and given the small sample size, we would perform a non-parametric test that would be equivalent to the paired \(t\)-test.

ApproximateSignedRankTest(
  intake.post,
  intake.pre
) |> pvalue |> r3
0.004
Example

We are going to use an example from Altman on the number of CD4 \(^+\) T cells and CD8 \(^+\) T cells in patients with Hodgkin’s disease or with disseminated malignancies (the Non-Hodgkin’s disease group).

hodgkin = rcopy(R"pubh::Hodgkin")
hodgkin |> schema
┌───────┬───────────────┬──────────────────────────────────┐
│ names │ scitypes      │ types                            │
├───────┼───────────────┼──────────────────────────────────┤
│ CD4   │ Count         │ Int64                            │
│ CD8   │ Count         │ Int64                            │
│ Group │ Multiclass{2} │ CategoricalValue{String, UInt32} │
└───────┴───────────────┴──────────────────────────────────┘
Exercise

Generate a new variable, named ratio that will contain the ratio of CD4 \(^+\) / CD8 \(^+\) T cells.

Code
@transform!(
  hodgkin,
  :ratio = :CD4 / :CD8
);
Exercise

Generate a table with descriptive statistics for ratio, stratified by Group.

Code
pubh.estat(@formula(ratio ~ Group), data=hodgkin) |> rcopy
2×9 DataFrame
Row Group N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 ratio Non-Hodgkin 20.0 1.1 3.49 2.12 2.15 0.73 0.34
2 Hodgkin 20.0 0.47 3.82 1.5 1.19 0.91 0.61

Let’s compare the distributions of the ratios with a QQ-Plot:

let
  hodg = DataFrames.combine(
    groupby(hodgkin, :Group), 
    :ratio=>(x->fit(Normal, x))=>:d
  )

  plot(
    hodg,
    x=:d, y=hodgkin.ratio, color=:Group,
    Stat.qq,
    Guide.xlabel("Theoretical quantiles"),
    Guide.ylabel("CD4 / CD8"))
end
Theoretical quantiles -1 0 1 2 3 4 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 -1.00 -0.98 -0.96 -0.94 -0.92 -0.90 -0.88 -0.86 -0.84 -0.82 -0.80 -0.78 -0.76 -0.74 -0.72 -0.70 -0.68 -0.66 -0.64 -0.62 -0.60 -0.58 -0.56 -0.54 -0.52 -0.50 -0.48 -0.46 -0.44 -0.42 -0.40 -0.38 -0.36 -0.34 -0.32 -0.30 -0.28 -0.26 -0.24 -0.22 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 2.02 2.04 2.06 2.08 2.10 2.12 2.14 2.16 2.18 2.20 2.22 2.24 2.26 2.28 2.30 2.32 2.34 2.36 2.38 2.40 2.42 2.44 2.46 2.48 2.50 2.52 2.54 2.56 2.58 2.60 2.62 2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 3.12 3.14 3.16 3.18 3.20 3.22 3.24 3.26 3.28 3.30 3.32 3.34 3.36 3.38 3.40 3.42 3.44 3.46 3.48 3.50 3.52 3.54 3.56 3.58 3.60 3.62 3.64 3.66 3.68 3.70 3.72 3.74 3.76 3.78 3.80 3.82 3.84 3.86 3.88 3.90 3.92 3.94 3.96 3.98 4.00 -2 0 2 4 Non-Hodgkin Hodgkin Group 3.4999758995067473.658692956349207 3.08920232367171633.425486434861435 2.8696685950639333.2658315474847734 2.71121106924384233.128292351078567 2.5837988771996543.0249287905346183 2.47532656251572062.850563011828936 2.37962811567551922.6980007383245543 2.29310603149832072.6516167918744324 2.21346259615291972.605665185970557 2.13913079618927072.4803445512820517 2.06898657654741852.19746359223301 2.00218998056567672.1889335558252427 1.93809127212557142.1856046195652175 1.87617232755426572.168494462674323 1.81600831569282572.106980925793498 1.75724160017314062.0076236730380015 1.69956327550933081.9795951848523483 1.64269959743712371.7795433145009416 1.58640159552210051.7775086612224698 1.5304367460855411.7689587973273944 1.47458192762038071.6408407572383072 1.41861707818382141.6121643776292998 1.36231907626879821.5023033553355336 1.3054553981965911.3954068638375088 1.24777707353278161.3624443078410358 1.1890103580130961.2886456618270719 1.12884634615165621.229603872859792 1.06692740158035051.203295218014219 1.00282869314024531.1743864926353316 0.93603209715850371.1437934491978612 0.86588787751665111.1286866759195002 0.79155607755300231.109557420296363 0.71191264220760121.015212839620153 0.62539055803040290.9551410824980217 0.52969211119020130.8697002024124255 0.421219796506268060.8269046620828442 0.293807604462079430.8149618065333708 0.1353500786419890.7591763976656847 -0.084183649965794640.6462260920221025 -0.4949572258008230.525891992250565 3.70712891645011043.658692956349207 3.3812430341474383.425486434861435 3.2070766635450733.2658315474847734 3.08136490006465553.128292351078567 2.9802828513081573.0249287905346183 2.89422669274360672.850563011828936 2.8183046407466712.6980007383245543 2.74966262626598252.6516167918744324 2.6864777654959332.605665185970557 2.62750687340518232.4803445512820517 2.57185818456255342.19746359223301 2.51886532207846072.1889335558252427 2.46801281991348542.1856046195652175 2.4188896263345152.168494462674323 2.371158702921352.106980925793498 2.32453632002285952.0076236730380015 2.27877740847506741.9795951848523483 2.2336647941549621.7795433145009416 2.1890009571769891.7775086612224698 2.1446014256358121.7689587973273944 2.1002891868084431.6408407572383072 2.05588965526726631.6121643776292998 2.01122581828929331.5023033553355336 1.96611320396918781.3954068638375088 1.92035429242139571.3624443078410358 1.87373190952290521.2886456618270719 1.82600098610974041.229603872859792 1.77687779253076971.203295218014219 1.72602529036579471.1743864926353316 1.67303242788170171.1437934491978612 1.61738373903907281.1286866759195002 1.55841284694832231.109557420296363 1.49522798617827271.015212839620153 1.42658597169758420.9551410824980217 1.35066391970064850.8697002024124255 1.26460776113609840.8269046620828442 1.16352571237959970.8149618065333708 1.03781394889918220.7591763976656847 0.86364757829681670.6462260920221025 0.53776169599414670.525891992250565 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 0 1 2 3 4 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 2.02 2.04 2.06 2.08 2.10 2.12 2.14 2.16 2.18 2.20 2.22 2.24 2.26 2.28 2.30 2.32 2.34 2.36 2.38 2.40 2.42 2.44 2.46 2.48 2.50 2.52 2.54 2.56 2.58 2.60 2.62 2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 3.12 3.14 3.16 3.18 3.20 3.22 3.24 3.26 3.28 3.30 3.32 3.34 3.36 3.38 3.40 3.42 3.44 3.46 3.48 3.50 3.52 3.54 3.56 3.58 3.60 3.62 3.64 3.66 3.68 3.70 3.72 3.74 3.76 3.78 3.80 3.82 3.84 3.86 3.88 3.90 3.92 3.94 3.96 3.98 4.00 0 5 CD4 / CD8
Figure 5: QQ-plot comparing the distributions of CD4/CD8 ratios between Hodking and Non-Hodgkin subjects.
Note

I know that for the normal, healthy population about 60 % of their T-cells is CD4 \(^+\) and about 40 % CD8 \(^+\) , i.e., a Ratio = 1.5. Given this, I know that the population who is showing abnormal levels is the group of non-Hodgkin’s lymphoma (see descriptive analysis). I would not be interested in knowing the confidence intervals of that difference.

Given that:

  • The sample size is relatively small.
  • The distribution of CD4 \(^+\) / CD8 \(^+\) T cells is not the same in the two groups.
  • Small changes (regardless of magnitude) in the distribution of T cell populations have significant biological consequences.

I would perform a non-parametric test. Once I know that this difference is statistically significant (i.e., very unlikely due to chance), I would conduct further studies to find out more about what is happening at a cellular and molecular level.

Would it be wrong to make a parametric test? Not at all, as long as the rationale and assumptions are clear. What is wrong it to perform both tests. We are not going to do that and perform only the Mann-Whitney test.

MannWhitneyUTest(
  vec_group(hodgkin, :ratio, :Group)...
) |> pvalue |> r3
0.007

4 ANOVA

Example

We will use a dataset on infant birth weight (in kg) and the smoking status of their mothers at the end of the first trimester.

smokew = read_rds("data/smokew.rds") |> rcopy
smokew |> schema
┌─────────┬───────────────┬──────────────────────────────────┐
│ names   │ scitypes      │ types                            │
├─────────┼───────────────┼──────────────────────────────────┤
│ bweight │ Continuous    │ Float64                          │
│ smoking │ Multiclass{4} │ CategoricalValue{String, UInt32} │
└─────────┴───────────────┴──────────────────────────────────┘

We can start with descriptive statistics.

pubh.estat(@formula(bweight ~ smoking), data=smokew) |> rcopy
4×9 DataFrame
Row smoking N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 bweight Non-smoker 7.0 2.82 4.18 3.45 3.41 0.44 0.13
2 Ex-smoker 5.0 2.64 3.73 3.29 3.32 0.41 0.13
3 Light-smoker 7.0 2.14 3.77 2.88 2.82 0.52 0.18
4 Heavy-smoker 8.0 2.23 3.23 2.73 2.73 0.33 0.12

Given the number of observations per group, we use a strip plot to compare the four groups graphically.

smokew_bst = pubh.gen_bst_df(
  @formula(bweight ~ smoking), 
  data=smokew
) |> rcopy
4×4 DataFrame
Row bweight LowerCI UpperCI smoking
Float64 Float64 Float64 Cat…
1 3.45 3.18 3.74 Non-smoker
2 3.29 2.94 3.58 Ex-smoker
3 2.88 2.56 3.22 Light-smoker
4 2.73 2.52 2.96 Heavy-smoker
strip_error(
  smokew,
  :smoking, :bweight,
  xlab = "Smoking status",
  ylab = "Birth weight (kg)"
)
Smoking status Non-smoker Ex-smoker Light-smoker Heavy-smoker 4,2.73375 32.8771428571428572 23.2939999999999996 13.447142857142857 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 2.0 2.5 3.0 3.5 4.0 4.5 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5 1.99 2.00 2.01 2.02 2.03 2.04 2.05 2.06 2.07 2.08 2.09 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22 2.23 2.24 2.25 2.26 2.27 2.28 2.29 2.30 2.31 2.32 2.33 2.34 2.35 2.36 2.37 2.38 2.39 2.40 2.41 2.42 2.43 2.44 2.45 2.46 2.47 2.48 2.49 2.50 2.51 2.52 2.53 2.54 2.55 2.56 2.57 2.58 2.59 2.60 2.61 2.62 2.63 2.64 2.65 2.66 2.67 2.68 2.69 2.70 2.71 2.72 2.73 2.74 2.75 2.76 2.77 2.78 2.79 2.80 2.81 2.82 2.83 2.84 2.85 2.86 2.87 2.88 2.89 2.90 2.91 2.92 2.93 2.94 2.95 2.96 2.97 2.98 2.99 3.00 3.01 3.02 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26 3.27 3.28 3.29 3.30 3.31 3.32 3.33 3.34 3.35 3.36 3.37 3.38 3.39 3.40 3.41 3.42 3.43 3.44 3.45 3.46 3.47 3.48 3.49 3.50 3.51 3.52 3.53 3.54 3.55 3.56 3.57 3.58 3.59 3.60 3.61 3.62 3.63 3.64 3.65 3.66 3.67 3.68 3.69 3.70 3.71 3.72 3.73 3.74 3.75 3.76 3.77 3.78 3.79 3.80 3.81 3.82 3.83 3.84 3.85 3.86 3.87 3.88 3.89 3.90 3.91 3.92 3.93 3.94 3.95 3.96 3.97 3.98 3.99 4.00 4.01 4.02 4.03 4.04 4.05 4.06 4.07 4.08 4.09 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20 4.21 4.22 4.23 4.24 4.25 4.26 4.27 4.28 4.29 4.30 4.31 4.32 4.33 4.34 4.35 4.36 4.37 4.38 4.39 4.40 4.41 4.42 4.43 4.44 4.45 4.46 4.47 4.48 4.49 4.50 2 3 4 5 Birth weight (kg)
Figure 6: Comparison of birth weights of babies born from mothers with different smoking status.

4.1 Initial assumptions

Normality can be tested using the Shapiro-Wilks test. The null hypothesis is that the distribution of the error is normal. We could look at the distribution of bweight for each group with QQ-plots. We will check for normality after fitting the model.

Homoscedasticity (homogeneity of variances) can be tested with Bartlett’s or Levene’s test of variance. The null hypothesis is that the variances are equal (homogeneous).

LeveneTest(
  vec_group(smokew, :bweight, :smoking)...
) |> pvalue |> r3
0.88

4.2 Model

We will make ANOVA after first fitting a linear model with lm:

model_smoke = lm(@formula(bweight ~ smoking), smokew)
anova(model_smoke)
Analysis of Variance

Type 1 test / F test

bweight ~ 1 + smoking

Table:
────────────────────────────────────────────────────────────
             DOF    Exp.SS  Mean Square    F value  Pr(>|F|)
────────────────────────────────────────────────────────────
(Intercept)    1  252.76       252.76    1392.6307    <1e-21
smoking        3    2.4085       0.8028     4.4234    0.0135
(Residuals)   23    4.1744       0.1815               
────────────────────────────────────────────────────────────
Interpretation

Not all groups of babies have the same mean birth weight. At least one of them is statistically different to another (\(p=0.014\)). From the descriptive statistics, we know that the cohort of babies born from non-smoker mothers have a mean birth weight significantly higher than those born from heavy-smoker mothers.

4.3 Post-hoc tests

So far, we know that there is evidence that at least the cohort of babies born from non-smoker mothers has a mean birth weight higher than those born from heavy-smoker mothers, but we do not know about any other comparison.

If we perform all possible paired t-test between the groups we would be increasing our error. To avoid that, we adjust our confidence intervals and p-values for multiple comparisons. There are several methods for making the adjustment.

We can use the function empairs to do the pairwise comparison and then adjust corresponding p-values with functions from MultipleTesting.

BH_adj(pvals) = MultipleTesting.adjust(PValues(pvals), BenjaminiHochberg());
empairs(model_smoke; dof=dof_residual, padjust=BH_adj)
6×6 DataFrame
Row smoking bweight err dof t Pr(>|t|)
String Float64 Float64 Float64 Float64 Float64
1 Non-smoker > Ex-smoker 0.153143 0.249453 23.0 0.613914 0.545297
2 Non-smoker > Light-smoker 0.57 0.227719 23.0 2.50309 0.0595751
3 Non-smoker > Heavy-smoker 0.713393 0.220488 23.0 3.23552 0.0219269
4 Ex-smoker > Light-smoker 0.416857 0.249453 23.0 1.67108 0.162388
5 Ex-smoker > Heavy-smoker 0.56025 0.24287 23.0 2.30679 0.0608319
6 Light-smoker > Heavy-smoker 0.143393 0.220488 23.0 0.650344 0.545297
Interpretation

We compared the birth weights of babies born from mothers of four different smoking status: non-smokers, ex-smokers, light-smokers and heavy-smokers with one-way ANOVA. We obtained a significant result (p = 0.014) that merited a post-hoc analysis. For the post-hoc analysis, we adjusted p-values for multiple comparisons by the method of Benjamini-Hochberg. After adjusting for multiple comparisons, the only statistical difference found was between babies born from non-smoker mothers and babies born from heavy-smoker mothers (p = 0.022). On average, babies born from non-smoker mothers had a birth weight of 0.71 kg higher than babies born from heavy-smoker mothers.

4.4 Diagnostics

smoke_perf = model_perf(model_smoke);

Normality:

resid_plot(smoke_perf)
Theoretical quantiles -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 -1.00 -0.99 -0.98 -0.97 -0.96 -0.95 -0.94 -0.93 -0.92 -0.91 -0.90 -0.89 -0.88 -0.87 -0.86 -0.85 -0.84 -0.83 -0.82 -0.81 -0.80 -0.79 -0.78 -0.77 -0.76 -0.75 -0.74 -0.73 -0.72 -0.71 -0.70 -0.69 -0.68 -0.67 -0.66 -0.65 -0.64 -0.63 -0.62 -0.61 -0.60 -0.59 -0.58 -0.57 -0.56 -0.55 -0.54 -0.53 -0.52 -0.51 -0.50 -0.49 -0.48 -0.47 -0.46 -0.45 -0.44 -0.43 -0.42 -0.41 -0.40 -0.39 -0.38 -0.37 -0.36 -0.35 -0.34 -0.33 -0.32 -0.31 -0.30 -0.29 -0.28 -0.27 -0.26 -0.25 -0.24 -0.23 -0.22 -0.21 -0.20 -0.19 -0.18 -0.17 -0.16 -0.15 -0.14 -0.13 -0.12 -0.11 -0.10 -0.09 -0.08 -0.07 -0.06 -0.05 -0.04 -0.03 -0.02 -0.01 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.31 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 0.40 0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.50 0.51 0.52 0.53 0.54 0.55 0.56 0.57 0.58 0.59 0.60 0.61 0.62 0.63 0.64 0.65 0.66 0.67 0.68 0.69 0.70 0.71 0.72 0.73 0.74 0.75 0.76 0.77 0.78 0.79 0.80 0.81 0.82 0.83 0.84 0.85 0.86 0.87 0.88 0.89 0.90 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1.00 -1 0 1 0.81996423781016360.8158201058201038 0.62645549519929540.627698412698411 0.52097490672593570.4717037037037033 0.4435873955066610.42002116402116363 0.38039128674787620.3806547619047611 0.32575326490682660.34635582010581867 0.27678286737117820.3055238095238072 0.231774706376042140.2182777777777776 0.18962055984885770.08624999999999927 0.149544485385670860.07732407407407339 0.110967717780698080.02342857142857094 0.073432724355876-0.00010582010582297404 0.036556639157821536-0.037883597883600656 -1.2335811384723961e-15-0.05714285714285827 -0.03655663915782396-0.05714285714285827 -0.07343272435587847-0.06349206349206363 -0.11096771778070051-0.08457142857143139 -0.1495444853856733-0.09277116402116486 -0.1896205598488601-0.1344907407407414 -0.23177470637604458-0.18527777777777862 -0.27678286737118063-0.22677248677248757 -0.32575326490682904-0.26994047619047673 -0.38039128674787864-0.29934523809524016 -0.44358739550666343-0.43093253968254114 -0.5209749067259382-0.5768716931216953 -0.6264554951992979-0.6420634920634934 -0.819964237810166-0.6971111111111115 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -1.0 -0.5 0.0 0.5 1.0 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 -0.740 -0.735 -0.730 -0.725 -0.720 -0.715 -0.710 -0.705 -0.700 -0.695 -0.690 -0.685 -0.680 -0.675 -0.670 -0.665 -0.660 -0.655 -0.650 -0.645 -0.640 -0.635 -0.630 -0.625 -0.620 -0.615 -0.610 -0.605 -0.600 -0.595 -0.590 -0.585 -0.580 -0.575 -0.570 -0.565 -0.560 -0.555 -0.550 -0.545 -0.540 -0.535 -0.530 -0.525 -0.520 -0.515 -0.510 -0.505 -0.500 -0.495 -0.490 -0.485 -0.480 -0.475 -0.470 -0.465 -0.460 -0.455 -0.450 -0.445 -0.440 -0.435 -0.430 -0.425 -0.420 -0.415 -0.410 -0.405 -0.400 -0.395 -0.390 -0.385 -0.380 -0.375 -0.370 -0.365 -0.360 -0.355 -0.350 -0.345 -0.340 -0.335 -0.330 -0.325 -0.320 -0.315 -0.310 -0.305 -0.300 -0.295 -0.290 -0.285 -0.280 -0.275 -0.270 -0.265 -0.260 -0.255 -0.250 -0.245 -0.240 -0.235 -0.230 -0.225 -0.220 -0.215 -0.210 -0.205 -0.200 -0.195 -0.190 -0.185 -0.180 -0.175 -0.170 -0.165 -0.160 -0.155 -0.150 -0.145 -0.140 -0.135 -0.130 -0.125 -0.120 -0.115 -0.110 -0.105 -0.100 -0.095 -0.090 -0.085 -0.080 -0.075 -0.070 -0.065 -0.060 -0.055 -0.050 -0.045 -0.040 -0.035 -0.030 -0.025 -0.020 -0.015 -0.010 -0.005 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060 0.065 0.070 0.075 0.080 0.085 0.090 0.095 0.100 0.105 0.110 0.115 0.120 0.125 0.130 0.135 0.140 0.145 0.150 0.155 0.160 0.165 0.170 0.175 0.180 0.185 0.190 0.195 0.200 0.205 0.210 0.215 0.220 0.225 0.230 0.235 0.240 0.245 0.250 0.255 0.260 0.265 0.270 0.275 0.280 0.285 0.290 0.295 0.300 0.305 0.310 0.315 0.320 0.325 0.330 0.335 0.340 0.345 0.350 0.355 0.360 0.365 0.370 0.375 0.380 0.385 0.390 0.395 0.400 0.405 0.410 0.415 0.420 0.425 0.430 0.435 0.440 0.445 0.450 0.455 0.460 0.465 0.470 0.475 0.480 0.485 0.490 0.495 0.500 0.505 0.510 0.515 0.520 0.525 0.530 0.535 0.540 0.545 0.550 0.555 0.560 0.565 0.570 0.575 0.580 0.585 0.590 0.595 0.600 0.605 0.610 0.615 0.620 0.625 0.630 0.635 0.640 0.645 0.650 0.655 0.660 0.665 0.670 0.675 0.680 0.685 0.690 0.695 0.700 0.705 0.710 0.715 0.720 0.725 0.730 0.735 0.740 0.745 0.750 0.755 0.760 0.765 0.770 0.775 0.780 0.785 0.790 0.795 0.800 0.805 0.810 0.815 0.820 -1 0 1 Residuals

Variance:

rvf_plot(smoke_perf)
Fitted values 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 2.700 2.705 2.710 2.715 2.720 2.725 2.730 2.735 2.740 2.745 2.750 2.755 2.760 2.765 2.770 2.775 2.780 2.785 2.790 2.795 2.800 2.805 2.810 2.815 2.820 2.825 2.830 2.835 2.840 2.845 2.850 2.855 2.860 2.865 2.870 2.875 2.880 2.885 2.890 2.895 2.900 2.905 2.910 2.915 2.920 2.925 2.930 2.935 2.940 2.945 2.950 2.955 2.960 2.965 2.970 2.975 2.980 2.985 2.990 2.995 3.000 3.005 3.010 3.015 3.020 3.025 3.030 3.035 3.040 3.045 3.050 3.055 3.060 3.065 3.070 3.075 3.080 3.085 3.090 3.095 3.100 3.105 3.110 3.115 3.120 3.125 3.130 3.135 3.140 3.145 3.150 3.155 3.160 3.165 3.170 3.175 3.180 3.185 3.190 3.195 3.200 3.205 3.210 3.215 3.220 3.225 3.230 3.235 3.240 3.245 3.250 3.255 3.260 3.265 3.270 3.275 3.280 3.285 3.290 3.295 3.300 3.305 3.310 3.315 3.320 3.325 3.330 3.335 3.340 3.345 3.350 3.355 3.360 3.365 3.370 3.375 3.380 3.385 3.390 3.395 3.400 3.405 3.410 3.415 3.420 3.425 3.430 3.435 3.440 3.445 3.450 3.455 3.460 3.465 3.470 3.475 3.480 3.485 3.490 3.495 3.500 2.5 3.0 3.5 2.7337500000000006-0.7081509894446134 2.7337500000000006-0.23397059122619257 2.73375000000000061.2384843295573247 2.73375000000000060.21525294392810027 2.7337500000000006-1.2572019768554172 2.7337500000000006-0.3587549065468304 2.73375000000000060.8890882466595406 2.73375000000000060.21525294392810027 2.877142857142858-0.14261064608072785 2.8771428571428580.9804481918050064 2.8771428571428582.2282913450113773 2.877142857142858-1.8396773344413917 2.877142857142858-0.5918341812350207 2.877142857142858-0.14261064608072785 2.877142857142858-0.49200672897851094 3.2940.6388956944416644 3.294-0.15972392361041254 3.2941.0881192295959583 3.2940.06488784396673387 3.294-1.63217884439393 3.447142857142860.0071305323040323654 3.447142857142860.8057501503561093 3.447142857142861.8289815359853328 3.44714285714286-0.21748123527311516 3.44714285714286-0.7665322226839177 3.44714285714286-1.565151840735996 3.44714285714286-0.09269691995247739 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -2 -1 0 1 2 3 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 -2.00 -1.98 -1.96 -1.94 -1.92 -1.90 -1.88 -1.86 -1.84 -1.82 -1.80 -1.78 -1.76 -1.74 -1.72 -1.70 -1.68 -1.66 -1.64 -1.62 -1.60 -1.58 -1.56 -1.54 -1.52 -1.50 -1.48 -1.46 -1.44 -1.42 -1.40 -1.38 -1.36 -1.34 -1.32 -1.30 -1.28 -1.26 -1.24 -1.22 -1.20 -1.18 -1.16 -1.14 -1.12 -1.10 -1.08 -1.06 -1.04 -1.02 -1.00 -0.98 -0.96 -0.94 -0.92 -0.90 -0.88 -0.86 -0.84 -0.82 -0.80 -0.78 -0.76 -0.74 -0.72 -0.70 -0.68 -0.66 -0.64 -0.62 -0.60 -0.58 -0.56 -0.54 -0.52 -0.50 -0.48 -0.46 -0.44 -0.42 -0.40 -0.38 -0.36 -0.34 -0.32 -0.30 -0.28 -0.26 -0.24 -0.22 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 2.02 2.04 2.06 2.08 2.10 2.12 2.14 2.16 2.18 2.20 2.22 2.24 2.26 2.28 2.30 2.32 2.34 2.36 2.38 2.40 2.42 2.44 2.46 2.48 2.50 2.52 2.54 2.56 2.58 2.60 2.62 2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 -3 0 3 Std residuals

Mean absolute error:

mean(abs.(smoke_perf.error)) |> r3
0.303

Mean absolute percentage error:

mape(smoke_perf) |> r3
0.004

Root mean square error:

rmse(smoke_perf) |> r3
0.393

4.4.1 Effects

In treatment coding of categorical variables (the default), the hypothesis for the coefficients for each level is against the reference level.

model_smoke
StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}, Vector{Int64}}}}, Matrix{Float64}}

bweight ~ 1 + smoking

Coefficients:
────────────────────────────────────────────────────────────────────────────────────
                           Coef.  Std. Error      t  Pr(>|t|)  Lower 95%   Upper 95%
────────────────────────────────────────────────────────────────────────────────────
(Intercept)             3.44714     0.161021  21.41    <1e-15   3.11404    3.78024
smoking: Ex-smoker     -0.153143    0.249453  -0.61    0.5453  -0.669176   0.362891
smoking: Light-smoker  -0.57        0.227719  -2.50    0.0199  -1.04107   -0.0989279
smoking: Heavy-smoker  -0.713393    0.220488  -3.24    0.0037  -1.16951   -0.257279
────────────────────────────────────────────────────────────────────────────────────

In effects coding of categorical variables, the hypothesis for the coefficients for each level is against the mean across all levels.

model_eff = lm(
  @formula(bweight ~ smoking), smokew;
  contrasts=Dict(:smoke => EffectsCoding())
)
StatsModels.TableRegressionModel{LinearModel{GLM.LmResp{Vector{Float64}}, GLM.DensePredChol{Float64, LinearAlgebra.CholeskyPivoted{Float64, Matrix{Float64}, Vector{Int64}}}}, Matrix{Float64}}

bweight ~ 1 + smoking

Coefficients:
────────────────────────────────────────────────────────────────────────────────────
                           Coef.  Std. Error      t  Pr(>|t|)  Lower 95%   Upper 95%
────────────────────────────────────────────────────────────────────────────────────
(Intercept)             3.44714     0.161021  21.41    <1e-15   3.11404    3.78024
smoking: Ex-smoker     -0.153143    0.249453  -0.61    0.5453  -0.669176   0.362891
smoking: Light-smoker  -0.57        0.227719  -2.50    0.0199  -1.04107   -0.0989279
smoking: Heavy-smoker  -0.713393    0.220488  -3.24    0.0037  -1.16951   -0.257279
────────────────────────────────────────────────────────────────────────────────────

To look at the effects, we first generate a reference grid:

model_des = Dict(:smoking => unique(smokew.smoking))
smoke_eff = effects(model_des, model_eff)
4×5 DataFrame
Row smoking bweight err lower upper
String Float64 Float64 Float64 Float64
1 Non-smoker 3.44714 0.161021 3.28612 3.60816
2 Ex-smoker 3.294 0.190523 3.10348 3.48452
3 Light-smoker 2.87714 0.161021 2.71612 3.03816
4 Heavy-smoker 2.73375 0.150622 2.58313 2.88437

We can use the estimated effects to get a nice visual of the data:

effect_plot(
  smoke_eff,
  :smoking, :bweight,
  xlab = "Smoking status",
  ylab = "Birth weight (kg)"
)
Smoking status Non-smoker Ex-smoker Light-smoker Heavy-smoker 42.7337500000000006 32.877142857142858 2,3.294 13.44714285714286 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 2.5 3.0 3.5 4.0 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 2.500 2.505 2.510 2.515 2.520 2.525 2.530 2.535 2.540 2.545 2.550 2.555 2.560 2.565 2.570 2.575 2.580 2.585 2.590 2.595 2.600 2.605 2.610 2.615 2.620 2.625 2.630 2.635 2.640 2.645 2.650 2.655 2.660 2.665 2.670 2.675 2.680 2.685 2.690 2.695 2.700 2.705 2.710 2.715 2.720 2.725 2.730 2.735 2.740 2.745 2.750 2.755 2.760 2.765 2.770 2.775 2.780 2.785 2.790 2.795 2.800 2.805 2.810 2.815 2.820 2.825 2.830 2.835 2.840 2.845 2.850 2.855 2.860 2.865 2.870 2.875 2.880 2.885 2.890 2.895 2.900 2.905 2.910 2.915 2.920 2.925 2.930 2.935 2.940 2.945 2.950 2.955 2.960 2.965 2.970 2.975 2.980 2.985 2.990 2.995 3.000 3.005 3.010 3.015 3.020 3.025 3.030 3.035 3.040 3.045 3.050 3.055 3.060 3.065 3.070 3.075 3.080 3.085 3.090 3.095 3.100 3.105 3.110 3.115 3.120 3.125 3.130 3.135 3.140 3.145 3.150 3.155 3.160 3.165 3.170 3.175 3.180 3.185 3.190 3.195 3.200 3.205 3.210 3.215 3.220 3.225 3.230 3.235 3.240 3.245 3.250 3.255 3.260 3.265 3.270 3.275 3.280 3.285 3.290 3.295 3.300 3.305 3.310 3.315 3.320 3.325 3.330 3.335 3.340 3.345 3.350 3.355 3.360 3.365 3.370 3.375 3.380 3.385 3.390 3.395 3.400 3.405 3.410 3.415 3.420 3.425 3.430 3.435 3.440 3.445 3.450 3.455 3.460 3.465 3.470 3.475 3.480 3.485 3.490 3.495 3.500 3.505 3.510 3.515 3.520 3.525 3.530 3.535 3.540 3.545 3.550 3.555 3.560 3.565 3.570 3.575 3.580 3.585 3.590 3.595 3.600 3.605 3.610 3.615 3.620 3.625 3.630 3.635 3.640 3.645 3.650 3.655 3.660 3.665 3.670 3.675 3.680 3.685 3.690 3.695 3.700 3.705 3.710 3.715 3.720 3.725 3.730 3.735 3.740 3.745 3.750 3.755 3.760 3.765 3.770 3.775 3.780 3.785 3.790 3.795 3.800 3.805 3.810 3.815 3.820 3.825 3.830 3.835 3.840 3.845 3.850 3.855 3.860 3.865 3.870 3.875 3.880 3.885 3.890 3.895 3.900 3.905 3.910 3.915 3.920 3.925 3.930 3.935 3.940 3.945 3.950 3.955 3.960 3.965 3.970 3.975 3.980 3.985 3.990 3.995 4.000 2 4 Birth weight (kg)
Figure 7: Effect of mother’s smoking status on babies’ birth weights. Circles indicate mean birth weights for each group and bars represent 95% CI around mean values.

4.5 Alternatives for non-Normal data

Example

The airquality dataset has daily readings on air quality values for May 1, 1973 (a Tuesday) to September 30, 1973.

air = rcopy(R"datasets::airquality") |> dropmissing
air.Month = recode(
  air.Month,
  5 => "May",
  6 => "Jun",
  7 => "Jul",
  8 => "Aug",
  9 => "Sep"
)
coerce!(air, :Month => Multiclass)
levels!(air.Month, unique(air.Month))
air |> schema
┌─────────┬───────────────┬──────────────────────────────────┐
│ names   │ scitypes      │ types                            │
├─────────┼───────────────┼──────────────────────────────────┤
│ Ozone   │ Count         │ Int64                            │
│ Solar_R │ Count         │ Int64                            │
│ Wind    │ Continuous    │ Float64                          │
│ Temp    │ Count         │ Int64                            │
│ Month   │ Multiclass{5} │ CategoricalValue{String, UInt32} │
│ Day     │ Count         │ Int64                            │
└─────────┴───────────────┴──────────────────────────────────┘
Exercise

Calculate descriptive statistics for Ozone by Month.

Code
pubh.estat(@formula(Ozone ~ Month), data=air) |> rcopy
5×9 DataFrame
Row Month N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 Ozone May 24.0 1.0 115.0 24.12 18.0 22.89 0.95
2 Jun 9.0 12.0 71.0 29.44 23.0 18.21 0.62
3 Jul 26.0 7.0 135.0 59.12 60.0 31.64 0.54
4 Aug 23.0 9.0 168.0 60.0 45.0 41.77 0.7
5 Sep 29.0 7.0 96.0 31.45 23.0 24.14 0.77
Interpretation

Look at the relative dispersion; the distribution of ozone is clearly not normal. The months of July and August have the highest median concentrations of ozone. We do not know yet, if the month with the highest median ozone concentration (July), is significantly different from the one with the lowest median ozone concentration (May). In June, ozone concentrations were recorded for only nine days.

Exercise

Use Levene’s test to test for homoscedasticity.

Code
LeveneTest(
  vec_group(air, :Ozone, :Month)...
) |> pvalue |> r3
0.002

4.5.1 Log-transformation

We can log-transform to make the distribution closer to Normal and the variance constant between groups.

We check for normality for each group.

let
  @transform!(air, :log_oz = log.(:Ozone))
  
  air_qq = DataFrames.combine(
    groupby(air, :Month),
    :log_oz => (x -> fit(Normal, x)) => :d
  )
  
  plot(
    air_qq,
    x = :d, y = air.log_oz, color = :Month,
    Stat.qq,
    Guide.xlabel("Normal quantiles"),
    Guide.ylabel("log (Ozone)")
  )
end
Normal quantiles 0 1 2 3 4 5 6 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 3.75 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 2.02 2.04 2.06 2.08 2.10 2.12 2.14 2.16 2.18 2.20 2.22 2.24 2.26 2.28 2.30 2.32 2.34 2.36 2.38 2.40 2.42 2.44 2.46 2.48 2.50 2.52 2.54 2.56 2.58 2.60 2.62 2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 3.12 3.14 3.16 3.18 3.20 3.22 3.24 3.26 3.28 3.30 3.32 3.34 3.36 3.38 3.40 3.42 3.44 3.46 3.48 3.50 3.52 3.54 3.56 3.58 3.60 3.62 3.64 3.66 3.68 3.70 3.72 3.74 3.76 3.78 3.80 3.82 3.84 3.86 3.88 3.90 3.92 3.94 3.96 3.98 4.00 4.02 4.04 4.06 4.08 4.10 4.12 4.14 4.16 4.18 4.20 4.22 4.24 4.26 4.28 4.30 4.32 4.34 4.36 4.38 4.40 4.42 4.44 4.46 4.48 4.50 4.52 4.54 4.56 4.58 4.60 4.62 4.64 4.66 4.68 4.70 4.72 4.74 4.76 4.78 4.80 4.82 4.84 4.86 4.88 4.90 4.92 4.94 4.96 4.98 5.00 5.02 5.04 5.06 5.08 5.10 5.12 5.14 5.16 5.18 5.20 5.22 5.24 5.26 5.28 5.30 5.32 5.34 5.36 5.38 5.40 5.42 5.44 5.46 5.48 5.50 5.52 5.54 5.56 5.58 5.60 5.62 5.64 5.66 5.68 5.70 5.72 5.74 5.76 5.78 5.80 5.82 5.84 5.86 5.88 5.90 5.92 5.94 5.96 5.98 6.00 0 10 May Jun Jul Aug Sep Month 4.93102668188735655.015604465411679 4.6684042520042514.856016205284561 4.5327652800662824.7881036548757585 4.4373843843837294.758620392057327 4.36245019705493854.7245083455604355 4.3000303742023524.69221498801395 4.2461092248932014.63471147773304 4.1983589663401124.574710978503383 4.15530401289454154.570323131740584 4.1159473311424784.542181709416076 4.0795821145183574.5018501267392645 4.0456881675572554.470408036555201 4.0138709339615564.442651256490317 3.98382365351564844.438173353596856 3.95530283257544254.42191590141321 3.92811176063954774.397821008492948 3.9020890912921864.377607062547147 3.87710072255162344.365086744543216 3.8530338913790784.352407691744289 3.8297927928034754.33956582783184 3.8072952728341234.318034363080674 3.7854702929016914.290459441148391 3.7642559586060124.282200651819261 3.7435979678466514.237226152458778 3.72344837514991864.170057271854061 3.70376459752465254.1588830833596715 3.68450860701540074.1547686477553825 3.665646269144124.134996851237387 3.64714679649196154.102765005189303 3.628982293989174.07753744390572 3.61112737786894484.0490929111750225 3.59355885425470063.942763564386124 3.57625544637218073.907836858866858 3.5591975616759463.8877335925389134 3.54236709194354573.8672179334920846 3.52574724075607373.846272609778353 3.5093223738493273.8248792412849806 3.4930778886575643.8066624897703196 3.47700010003517073.8032207010362207 3.4610761396702093.784189633918261 3.4452938671277123.784189633918261 3.4296417908024153.775282913729114 3.41410899733769523.710680139103634 3.3986850882931433.68614239379131 3.38336012302776463.6635616461296463 3.3681245669173293.658818967437266 3.352969244149123.6109179126442243 3.3378852944403233.608943211801838 3.3228641331113523.58351893845611 3.30789741401573073.5819961883498017 3.29297699488615253.554042316585213 3.27809490470404443.5241758535597127 3.263243312738873.4657359027997265 3.24841449893480763.4657359027997265 3.23360082534730743.465449878490586 3.2187947083510713.4339872044851463 3.20398859135483473.4014927854713712 3.18917491776733453.4011973816621555 3.1743461039632723.3682120881398707 3.15949451199809773.3334690622404746 3.14461242181598973.332204510175204 3.12969200268641153.2978026846081603 3.114725283590793.1854815893533033 3.09970412226181943.1780538303479458 3.08462017255302223.1389449954766193 3.0694648497848133.1354942159291497 3.05422929367437763.1354942159291497 3.0389043284089993.1354942159291497 3.0234804193644473.1354942159291497 3.00794762589972733.1354942159291497 2.99229554957443033.097049448300321 2.97651327703193273.0512280255626867 2.96058931666697143.044522437723423 2.9445115280445783.044522437723423 2.92826704285281553.044522437723423 2.91184217594606843.0045232941250597 2.89522232475859642.995732273553991 2.8783918550261962.995732273553991 2.86133397032996142.995732273553991 2.84403056244744162.9555294211961813 2.8264620388331982.9025490599840644 2.80860712271297252.8903717578961645 2.79044262021018062.8903717578961645 2.7719431475580222.8903717578961645 2.75308080968674142.803360866690548 2.73382481917748972.772588722239781 2.71414104155222362.772588722239781 2.6939914488554912.772588722239781 2.67333345809613032.6787558517468737 2.6521191238004512.6390573296152584 2.6302941438680192.6390573296152584 2.6077966238986672.6390573296152584 2.5845555253230642.5896520148461106 2.56048869415051872.5649493574615367 2.5355003254099562.5649493574615367 2.50947765606259442.5649493574615367 2.48228658412669972.5144719742439916 2.45376576318649382.4849066497880004 2.42371848274058582.4316023828033626 2.3919012491448862.3978952727983707 2.35800730218378572.3978952727983707 2.3216420855596642.342083005345388 2.28228540380760062.2418366875697138 2.23923045036203042.1972245773362196 2.19148019180894242.1972245773362196 2.137559042499792.132496963146675 2.07513921964720362.007262410531445 2.0002050323184131.9459101490553132 1.90482413663586051.8653629469834123 1.76918516469789131.583547656956295 1.50656273481478830.6869026113657116 5.9022237661511615.015604465411679 5.5829670612341424.856016205284561 5.41807767513516054.7881036548757585 5.302127980810884.758620392057327 5.2110343078794684.7245083455604355 5.1351537101167774.69221498801395 5.0696045166931964.63471147773304 5.0115569606966264.574710978503383 4.9592172478491494.570323131740584 4.9113733354376154.542181709416076 4.8671659947387334.5018501267392645 4.8259628506346694.470408036555201 4.7872842611407734.442651256490317 4.7507573134952574.438173353596856 4.7160860048833034.42191590141321 4.6830312043726264.397821008492948 4.6513967713476874.377607062547147 4.6210196848630554.365086744543216 4.5917628646388224.352407691744289 4.5635098454538224.33956582783184 4.5361607568880934.318034363080674 4.5096292409729844.290459441148391 4.4838400558403154.282200651819261 4.45872718919379454.237226152458778 4.434232356164334.170057271854061 4.4103037907795724.1588830833596715 4.3868952643919624.1547686477553825 4.3639652814560144.134996851237387 4.3414764152755464.102765005189303 4.3193947552357724.07753744390572 4.2976894435849354.0490929111750225 4.2763322847092173.942763564386124 4.2552974135184083.907836858866858 4.2345610123534413.8877335925389134 4.214101067971073.8672179334920846 4.1938971618207643.846272609778353 4.1739302881242473.8248792412849806 4.1541826952864833.8066624897703196 4.1346377469733393.8032207010362207 4.1152797998337523.784189633918261 4.0960940953596043.784189633918261 4.077066663791983.775282913729114 4.0581842383193723.710680139103634 4.0394341780876723.68614239379131 4.0208043987661943.6635616461296463 4.002283309598143.658818967437266 3.98385975601558463.6109179126442243 3.96552296702422563.608943211801838 3.9472625066665613.58351893845611 3.92906822895763243.5819961883498017 3.91093023575800253.554042316585213 3.8928388371065913.5241758535597127 3.87478451358333283.4657359027997265 3.85675788030977663.4657359027997265 3.8387496522259583.465449878490586 3.82075061030505973.4339872044851463 3.80275156838416133.4014927854713712 3.78474334030034283.4011973816621555 3.76671670702678663.3682120881398707 3.74866238350352843.3334690622404746 3.7305709848521173.332204510175204 3.7124329916524873.2978026846081603 3.69423871394355843.1854815893533033 3.6759782535858943.1780538303479458 3.65764146459453433.1389449954766193 3.63921791101197873.1354942159291497 3.6206968218439253.1354942159291497 3.60206704252244773.1354942159291497 3.5833169822907483.1354942159291497 3.56443455681813953.1354942159291497 3.54540712525051533.097049448300321 3.52622142077636673.0512280255626867 3.50686347363678053.044522437723423 3.48731852532363633.044522437723423 3.46757093248587373.044522437723423 3.44760405878935573.0045232941250597 3.42740015263904942.995732273553991 3.40694020825667782.995732273553991 3.38620380709171132.995732273553991 3.36516893590090232.9555294211961813 3.34381177702518472.9025490599840644 3.32210646537434772.8903717578961645 3.30002480533457332.8903717578961645 3.27753593915410462.8903717578961645 3.25460595621815772.803360866690548 3.2311974298305472.772588722239781 3.20726886444578962.772588722239781 3.18277403141632532.772588722239781 3.1576611647698042.6787558517468737 3.13187197963713572.6390573296152584 3.1053404637220262.6390573296152584 3.07799137515629662.6390573296152584 3.04973835597129742.5896520148461106 3.02048153574706472.5649493574615367 2.99010444926243222.5649493574615367 2.95847001623749332.5649493574615367 2.9254152157268162.5144719742439916 2.8907439071148622.4849066497880004 2.85421695946934672.4316023828033626 2.81553836997545042.3978952727983707 2.77433522587138632.3978952727983707 2.73012788517250462.342083005345388 2.68228397276097042.2418366875697138 2.62994425991349352.1972245773362196 2.5718967039169242.1972245773362196 2.50634751049334262.132496963146675 2.4304669127306512.007262410531445 2.33937323979923931.9459101490553132 2.2234235454749591.8653629469834123 2.0585341593759781.583547656956295 1.73927745445896150.6869026113657116 5.7430270097965085.015604465411679 5.4578637369688364.856016205284561 5.310582892838244.7881036548757585 5.2070154694838154.758620392057327 5.125649692330394.7245083455604355 5.0578723856399834.69221498801395 4.9993231923419424.63471147773304 4.947474536408684.574710978503383 4.9007241829013784.570323131740584 4.85798952318076354.542181709416076 4.8185030855819734.5018501267392645 4.7817000263429854.470408036555201 4.7471519244595684.442651256490317 4.7145256905337764.438173353596856 4.6835569318993324.42191590141321 4.654032054422634.397821008492948 4.6257758633497314.377607062547147 4.5986427467463434.365086744543216 4.572510263139334.352407691744289 4.5472743846316214.33956582783184 4.5228459059680824.318034363080674 4.4991476913498534.290459441148391 4.4761125339898794.282200651819261 4.45368147104675854.237226152458778 4.4318024418941374.170057271854061 4.4104292086492414.1588830833596715 4.3895204794230254.1547686477553825 4.3690391899804214.134996851237387 4.3489519104231364.102765005189303 4.3292283514518474.07753744390572 4.3098409506149374.0490929111750225 4.2907645233089453.942763564386124 4.2719759665773373.907836858866858 4.2534540062494983.8877335925389134 4.2351789798770193.8672179334920846 4.217132649406933.846272609778353 4.1992980386885253.8248792412849806 4.1816592918200813.8066624897703196 4.1642015490620323.8032207010362207 4.1469108376171953.784189633918261 4.129773975038933.784189633918261 4.1127784833992523.775282913729114 4.0959125126498113.710680139103634 4.0791647718536473.68614239379131 4.0625244671660743.6635616461296463 4.0459812456075223.658818967437266 4.0295251438066753.6109179126442243 4.0131465410040013.608943211801838 3.99683611569817953.58351893845611 3.9805848053942593.5819961883498017 3.96438376897537833.554042316585213 3.94822435127166043.5241758535597127 3.9320980494421663.4657359027997265 3.91599648081986333.4657359027997265 3.89991135189659273.465449878490586 3.8838344281456573.4339872044851463 3.8677575043947213.4014927854713712 3.85167237547145063.4011973816621555 3.8355708068491483.3682120881398707 3.8194445050196533.3334690622404746 3.80328508731593563.332204510175204 3.78708405089705473.2978026846081603 3.77083274059313433.1854815893533033 3.75452231528731333.1780538303479458 3.7381437124846393.1389449954766193 3.72168761068379133.1354942159291497 3.705144389125243.1354942159291497 3.6885040844376673.1354942159291497 3.67175634364150353.1354942159291497 3.6548903728920623.1354942159291497 3.6378948812523843.097049448300321 3.62075801867411863.0512280255626867 3.60346730722928273.044522437723423 3.5860095644712333.044522437723423 3.56837081760278933.044522437723423 3.5505362068843843.0045232941250597 3.53248987641429452.995732273553991 3.51421485004181562.995732273553991 3.4956928897139772.995732273553991 3.4769043329823692.9555294211961813 3.4578279056763772.9025490599840644 3.4384405048394672.8903717578961645 3.41871694586817882.8903717578961645 3.3986296663108932.8903717578961645 3.37814837686828942.803360866690548 3.3572396476420732.772588722239781 3.33586641439717772.772588722239781 3.3139873852445562.772588722239781 3.29155632230143572.6787558517468737 3.2685211649414612.6390573296152584 3.2448229503232322.6390573296152584 3.2203944716596932.6390573296152584 3.19515859315198372.5896520148461106 3.1690261095449712.5649493574615367 3.14189299294158262.5649493574615367 3.11363680186868352.5649493574615367 3.08411192439198172.5144719742439916 3.0531431657575382.4849066497880004 3.0205169318317462.4316023828033626 2.98596882994832852.3978952727983707 2.949165770709342.3978952727983707 2.9096793331105512.342083005345388 2.8669446733899362.2418366875697138 2.8201943198826342.1972245773362196 2.76834566394937242.1972245773362196 2.70979647065133022.132496963146675 2.64201916396092432.007262410531445 2.5606533868074991.9459101490553132 2.4570859634530741.8653629469834123 2.3098051193224781.583547656956295 2.02464184649480930.6869026113657116 4.6169115961276295.015604465411679 4.40521042119835654.856016205284561 4.2958712084805094.7881036548757585 4.2189842202259634.758620392057327 4.15857941966518754.7245083455604355 4.1082625071202414.69221498801395 4.0647964152583814.63471147773304 4.0263047074352344.574710978503383 3.9915979071249584.570323131740584 3.95987230100693574.542181709416076 3.9305581289870114.5018501267392645 3.90323605905623874.470408036555201 3.87758803752888164.442651256490317 3.85336678355350954.438173353596856 3.83037601575931764.42191590141321 3.8084571648866884.397821008492948 3.78748016883161184.377607062547147 3.76733692735639724.365086744543216 3.74793654164554954.352407691744289 3.72920178286164774.33956582783184 3.71106642628671064.318034363080674 3.69347320739615274.290459441148391 3.6763722328231254.282200651819261 3.65971972938927434.237226152458778 3.64347704802293664.170057271854061 3.6276098623748184.1588830833596715 3.6120875179313144.1547686477553825 3.59688249872925874.134996851237387 3.58196998688563144.102765005189303 3.567327496053614.07753744390572 3.5529345642594834.0490929111750225 3.53877249481033653.942763564386124 3.5248241363984513.907836858866858 3.51107369538087173.8877335925389134 3.4975065746343823.8672179334920846 3.48410923448676573.846272609778353 3.4708690720841753.8248792412849806 3.4577743162297413.8066624897703196 3.4448139352632713.8032207010362207 3.43197755597803633.784189633918261 3.4192553919123553.784189633918261 3.406638179629223.775282913729114 3.39411712182056843.710680139103634 3.3816838362547193.68614239379131 3.36933030973425043.6635616461296463 3.35704885635376153.658818967437266 3.34483207944749953.6109179126442243 3.332672836699853.608943211801838 3.32056420796025353.58351893845611 3.3084994653608123.5819961883498017 3.2964720453815863.554042316585213 3.2844755225470443.5241758535597127 3.2725035844684993.4657359027997265 3.26055000797267263.4657359027997265 3.2486086360765693.465449878490586 3.2366733555841963.4339872044851463 3.2247380750918233.4014927854713712 3.21279670319571943.4011973816621555 3.2008431266998933.3682120881398707 3.1888711886213483.3334690622404746 3.1768746657868063.332204510175204 3.164847245807583.2978026846081603 3.15278250320813853.1854815893533033 3.14067387446854253.1780538303479458 3.12851463172089253.1389449954766193 3.11629785481463053.1354942159291497 3.10401640143414163.1354942159291497 3.0916628749136733.1354942159291497 3.07922958934782363.1354942159291497 3.0667085315391723.1354942159291497 3.05409131925603643.097049448300321 3.04136915519035523.0512280255626867 3.0285327759051213.044522437723423 3.0155723949386513.044522437723423 3.0024776390842173.044522437723423 2.98923747668162683.0045232941250597 2.975840136534012.995732273553991 2.96227301578752032.995732273553991 2.9485225747699412.995732273553991 2.93457421635805552.9555294211961813 2.9204121469089092.9025490599840644 2.90601921511478262.8903717578961645 2.89137672428276062.8903717578961645 2.87646421243913332.8903717578961645 2.8612591932370782.803360866690548 2.8457368487935742.772588722239781 2.82986966314545542.772588722239781 2.81362698177911772.772588722239781 2.7969744783452672.6787558517468737 2.77987350377223932.6390573296152584 2.76228028488168142.6390573296152584 2.74414492830674432.6390573296152584 2.72541016952284252.5896520148461106 2.70600978381199482.5649493574615367 2.68586654233678022.5649493574615367 2.6648895462817042.5649493574615367 2.64297069540907442.5144719742439916 2.61997992761488252.4849066497880004 2.59575867363951042.4316023828033626 2.57011065211215332.3978952727983707 2.5427885821813812.3978952727983707 2.51347441016145632.342083005345388 2.4817488040434342.2418366875697138 2.44704200373315842.1972245773362196 2.4085502959100122.1972245773362196 2.36508420404815032.132496963146675 2.3147672915032042.007262410531445 2.25436249094242941.9459101490553132 2.17747550268788271.8653629469834123 2.06813628997003551.583547656956295 1.85643511504076520.6869026113657116 5.2397477418035735.015604465411679 4.8696024967875754.856016205284561 4.6784302480042364.7881036548757585 4.5439985278424134.758620392057327 4.4383848026886424.7245083455604355 4.3504090702505954.69221498801395 4.27441153633293254.63471147773304 4.2071113778522684.574710978503383 4.1464288752215664.570323131740584 4.0909587898346534.542181709416076 4.03970493454366954.5018501267392645 3.99193413557550824.470408036555201 3.9470902974834864.442651256490317 3.90474106635473064.438173353596856 3.86454325743312134.42191590141321 3.826219623048814.397821008492948 3.7895427583734024.377607062547147 3.7543236574224954.365086744543216 3.72040338973977754.352407691744289 3.68764692590484744.33956582783184 3.6559384764589474.318034363080674 3.62517791823794874.290459441148391 3.59527801605053554.282200651819261 3.5661622354427444.237226152458778 3.53776300111601664.170057271854061 3.51002029576069944.1588830833596715 3.4828805220245224.1547686477553825 3.45629557009925264.134996851237387 3.43022204758806474.102765005189303 3.40462063862812864.07753744390572 3.379455566836654.0490929111750225 3.35469414230540063.942763564386124 3.33030637712807833.907836858866858 3.30626465718377333.8877335925389134 3.2825434603857453.8672179334920846 3.25911911352910183.846272609778353 3.23596958137277023.8248792412849806 3.21307428277188433.8066624897703196 3.1904139296116463.8032207010362207 3.1679703850387883.784189633918261 3.14572653808423743.784189633918261 3.1236661922523343.775282913729114 3.10177396604248353.710680139103634 3.0800352036871833.68614239379131 3.058435894650483.6635616461296463 3.03696260064447233.658818967437266 3.01560238909730053.6109179126442243 2.994342772151193.608943211801838 2.9731716503890033.58351893845611 2.9520772605868823.5819961883498017 2.9310481268723043.554042316585213 2.9100730147340973.5241758535597127 2.88914088738581983.4657359027997265 2.8682408640281643.4657359027997265 2.84736217959106953.465449878490586 2.8264941455631053.4339872044851463 2.80562611153514043.4014927854713712 2.7847474270980463.4011973816621555 2.76384740374038973.3682120881398707 2.74291527639211273.3334690622404746 2.7219401642539063.332204510175204 2.7009110305393283.2978026846081603 2.6798166407372073.1854815893533033 2.658645518975023.1780538303479458 2.6373859020289093.1389449954766193 2.61602569048173763.1354942159291497 2.59455239647572983.1354942159291497 2.57295308743902673.1354942159291497 2.55121432508372653.1354942159291497 2.52932209887387633.1354942159291497 2.50726175304197253.097049448300321 2.4850179060874223.0512280255626867 2.4625743615145643.044522437723423 2.43991400835432563.044522437723423 2.417018709753443.044522437723423 2.3938691775971083.0045232941250597 2.3704448307404652.995732273553991 2.34672363394243672.995732273553991 2.32268191399813162.995732273553991 2.29829414882080932.9555294211961813 2.273532724289562.9025490599840644 2.24836765249808132.8903717578961645 2.22276624353814522.8903717578961645 2.19669272102695732.8903717578961645 2.1701077691016882.803360866690548 2.14296799536551052.772588722239781 2.11522529001019332.772588722239781 2.08682605568346662.772588722239781 2.05771027507567442.6787558517468737 2.02781037288826132.6390573296152584 1.9970498146672632.6390573296152584 1.96534136522136252.6390573296152584 1.93258490138643222.5896520148461106 1.8986646337037152.5649493574615367 1.8634455327528082.5649493574615367 1.82676866807739982.5649493574615367 1.78844503369308882.5144719742439916 1.74824722477147932.4849066497880004 1.7058979936427242.4316023828033626 1.66105415555070122.3978952727983707 1.61328335658254042.3978952727983707 1.56202950129155642.342083005345388 1.50655941590464432.2418366875697138 1.44587691327394222.1972245773362196 1.37857675479327772.1972245773362196 1.30257922087561462.132496963146675 1.21460348843756832.007262410531445 1.10898976328379731.9459101490553132 0.97455804312197311.8653629469834123 0.78338579433863581.583547656956295 0.413240549322640760.6869026113657116 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 0 1 2 3 4 5 6 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 3.75 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 6.00 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 2.02 2.04 2.06 2.08 2.10 2.12 2.14 2.16 2.18 2.20 2.22 2.24 2.26 2.28 2.30 2.32 2.34 2.36 2.38 2.40 2.42 2.44 2.46 2.48 2.50 2.52 2.54 2.56 2.58 2.60 2.62 2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 3.12 3.14 3.16 3.18 3.20 3.22 3.24 3.26 3.28 3.30 3.32 3.34 3.36 3.38 3.40 3.42 3.44 3.46 3.48 3.50 3.52 3.54 3.56 3.58 3.60 3.62 3.64 3.66 3.68 3.70 3.72 3.74 3.76 3.78 3.80 3.82 3.84 3.86 3.88 3.90 3.92 3.94 3.96 3.98 4.00 4.02 4.04 4.06 4.08 4.10 4.12 4.14 4.16 4.18 4.20 4.22 4.24 4.26 4.28 4.30 4.32 4.34 4.36 4.38 4.40 4.42 4.44 4.46 4.48 4.50 4.52 4.54 4.56 4.58 4.60 4.62 4.64 4.66 4.68 4.70 4.72 4.74 4.76 4.78 4.80 4.82 4.84 4.86 4.88 4.90 4.92 4.94 4.96 4.98 5.00 5.02 5.04 5.06 5.08 5.10 5.12 5.14 5.16 5.18 5.20 5.22 5.24 5.26 5.28 5.30 5.32 5.34 5.36 5.38 5.40 5.42 5.44 5.46 5.48 5.50 5.52 5.54 5.56 5.58 5.60 5.62 5.64 5.66 5.68 5.70 5.72 5.74 5.76 5.78 5.80 5.82 5.84 5.86 5.88 5.90 5.92 5.94 5.96 5.98 6.00 0 10 log (Ozone)
Figure 8: QQ-plots for log(Ozone) from May to September (New York, 1973).

Normality seems to be good enough, though we will check the distribution of the residuals later. What about homoscedasticity?

LeveneTest(
  vec_group(air, :log_oz, :Month)...
) |> pvalue |> r3
0.553

We can proceed to fit an ANOVA model to our data.

model_air = lm(
  @formula(log_oz ~ Month), air;
  contrasts=Dict(:Month => EffectsCoding())
)
anova(model_air)
Analysis of Variance

Type 1 test / F test

log_oz ~ 1 + Month

Table:
───────────────────────────────────────────────────────────
             DOF   Exp.SS  Mean Square    F value  Pr(>|F|)
───────────────────────────────────────────────────────────
(Intercept)    1  1295.21    1295.21    2170.4984    <1e-71
Month          4    19.22       4.8040     8.0506    <1e-04
(Residuals)  106    63.25       0.5967               
───────────────────────────────────────────────────────────

Diagnostics:

air_perf = model_perf(model_air);

Normality:

resid_plot(air_perf)
Theoretical quantiles -3 -2 -1 0 1 2 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 -3.00 -2.98 -2.96 -2.94 -2.92 -2.90 -2.88 -2.86 -2.84 -2.82 -2.80 -2.78 -2.76 -2.74 -2.72 -2.70 -2.68 -2.66 -2.64 -2.62 -2.60 -2.58 -2.56 -2.54 -2.52 -2.50 -2.48 -2.46 -2.44 -2.42 -2.40 -2.38 -2.36 -2.34 -2.32 -2.30 -2.28 -2.26 -2.24 -2.22 -2.20 -2.18 -2.16 -2.14 -2.12 -2.10 -2.08 -2.06 -2.04 -2.02 -2.00 -1.98 -1.96 -1.94 -1.92 -1.90 -1.88 -1.86 -1.84 -1.82 -1.80 -1.78 -1.76 -1.74 -1.72 -1.70 -1.68 -1.66 -1.64 -1.62 -1.60 -1.58 -1.56 -1.54 -1.52 -1.50 -1.48 -1.46 -1.44 -1.42 -1.40 -1.38 -1.36 -1.34 -1.32 -1.30 -1.28 -1.26 -1.24 -1.22 -1.20 -1.18 -1.16 -1.14 -1.12 -1.10 -1.08 -1.06 -1.04 -1.02 -1.00 -0.98 -0.96 -0.94 -0.92 -0.90 -0.88 -0.86 -0.84 -0.82 -0.80 -0.78 -0.76 -0.74 -0.72 -0.70 -0.68 -0.66 -0.64 -0.62 -0.60 -0.58 -0.56 -0.54 -0.52 -0.50 -0.48 -0.46 -0.44 -0.42 -0.40 -0.38 -0.36 -0.34 -0.32 -0.30 -0.28 -0.26 -0.24 -0.22 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74 1.76 1.78 1.80 1.82 1.84 1.86 1.88 1.90 1.92 1.94 1.96 1.98 2.00 -3 0 3 1.9715468957509851.6345762937678432 1.66915069921152971.3249555898104355 1.512969411745641.297890177571908 1.40314322481344011.2198500652737292 1.31686036904560931.1074752114682398 1.24498715917663421.0510979709323653 1.1828997340036161.023990824276462 1.1279177715845771.0049344407297123 1.07834221107392760.9819569367670603 1.03302501986701080.9676388020257372 0.99115234662138450.9244518142878747 0.95212521306196650.8841651347558946 0.91548930794133610.8481836552181834 0.88089141267686370.7930475436660165 0.84805115688325190.7743034252638885 0.81674203855354490.736427166284015 0.78677827560325660.696707790592535 0.75800545650994890.690876550357725 0.73029374044451480.6832129533808433 0.70353281291905940.658595812770715 0.67762807784623810.636754277635566 0.65249773797368790.6284576104409076 0.62807052508869050.6183822939118232 0.60428391312048070.6100029091435926 0.58108269532646640.6000318501811936 0.55841783958639050.5721874764958452 0.53624555866891740.5636763235901322 0.51452654848134050.5497528980894358 0.49322535889715880.5197464731122192 0.47230987018082770.5072209857490255 0.451750854232759740.4953591474726243 0.43152160449911830.4821745932990864 0.41159762187062080.4676910808062858 0.39195634654068820.45341406154967806 0.3725769278241850.4169286652914522 0.35344002551016610.3725291828339048 0.334527637548948430.36282635767463545 0.315822949838469370.3463982937703824 0.29731020463867420.30900007036604515 0.27897458475139820.3041061726864729 0.260802111091305470.27504865521401367 0.24277955166703330.27306237595677096 0.224894340310734380.25900592851265136 0.20713450375405050.25572244017882123 0.189488595861063790.24496894811864642 0.171945638003237950.2240361549224652 0.154495064705008320.19278677177771236 0.1371266738072360.17867076708342297 0.119830580493698210.12982298354108443 0.10259717460675680.11769969675040584 0.085417080745134520.1113377067165489 0.068281120691647530.06728202275639275 0.051180277763555970.06387761233306 0.034105662714353450.06323456665647334 0.0170484808444311550.028006570910258294 2.6205264184846036e-160.00798586996496864 -0.017048480844430627-0.012447657893597953 -0.034105662714352815-0.014061909664524766 -0.05118027776355556-0.0359536019043111 -0.0682811206916471-0.036560976386799116 -0.08541708074513399-0.04055259414653405 -0.10259717460675627-0.040740878003125314 -0.11983058049369769-0.05307522677159919 -0.13712667380723542-0.08118192888328361 -0.15449506470500796-0.08330049242192139 -0.17194563800323756-0.0995684507151244 -0.18948859586106329-0.15163844102474666 -0.20713450375405-0.17242542668686603 -0.22489434031073388-0.17427227062764805 -0.24277955166703263-0.18577642284439774 -0.2608021110913051-0.1874368159478461 -0.27897458475139786-0.19147140767512513 -0.29731020463867364-0.1945255239875221 -0.3158229498384688-0.21850446818648153 -0.33452763754894765-0.2230624347970802 -0.35344002551016535-0.2377197041503621 -0.37257692782418467-0.3118723266988821 -0.3919563465406877-0.32842295045490655 -0.41159762187062027-0.3284732263622734 -0.43152160449911775-0.3387548192088642 -0.4517508542327589-0.3765886513053168 -0.472309870180827-0.41075876761911645 -0.49322535889715835-0.4260447342034035 -0.51452654848134-0.42859887276473385 -0.536245558668917-0.42859887276473385 -0.55841783958639-0.4414473068284369 -0.581082695326466-0.47672138360214855 -0.6042839131204799-0.5534480011196811 -0.6280705250886901-0.5797373787358127 -0.6524977379736875-0.5854674158729725 -0.6776280778462377-0.6330825346175775 -0.703532812919059-0.6538453508895343 -0.7302937404445143-0.6538453508895343 -0.7580054565099487-0.6656033801509583 -0.7867782756032562-0.6805017686653083 -0.8167420385535444-0.7136895037680653 -0.8480511568832515-0.7406460964562641 -0.8808914126768632-0.7499829915588714 -0.9154893079413354-0.7667521088719612 -0.9521252130619664-0.8437557112723759 -0.9911523466213841-0.9674614919243713 -1.0330250198670103-1.0292790989951477 -1.0783422110739271-1.042476492107434 -1.1279177715845765-1.0839662171099427 -1.1828997340036151-1.2015304708623868 -1.2449871591766342-1.3648325659083238 -1.3168603690456089-1.5164427848261317 -1.4031432248134397-1.6037207330904992 -1.5129694117456396-1.6235260329688406 -1.6691506992115288-1.7849737809771806 -1.9715468957509812-2.386211779292818 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -3 -2 -1 0 1 2 -3.0 -2.8 -2.6 -2.4 -2.2 -2.0 -1.8 -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 -2.84 -2.82 -2.80 -2.78 -2.76 -2.74 -2.72 -2.70 -2.68 -2.66 -2.64 -2.62 -2.60 -2.58 -2.56 -2.54 -2.52 -2.50 -2.48 -2.46 -2.44 -2.42 -2.40 -2.38 -2.36 -2.34 -2.32 -2.30 -2.28 -2.26 -2.24 -2.22 -2.20 -2.18 -2.16 -2.14 -2.12 -2.10 -2.08 -2.06 -2.04 -2.02 -2.00 -1.98 -1.96 -1.94 -1.92 -1.90 -1.88 -1.86 -1.84 -1.82 -1.80 -1.78 -1.76 -1.74 -1.72 -1.70 -1.68 -1.66 -1.64 -1.62 -1.60 -1.58 -1.56 -1.54 -1.52 -1.50 -1.48 -1.46 -1.44 -1.42 -1.40 -1.38 -1.36 -1.34 -1.32 -1.30 -1.28 -1.26 -1.24 -1.22 -1.20 -1.18 -1.16 -1.14 -1.12 -1.10 -1.08 -1.06 -1.04 -1.02 -1.00 -0.98 -0.96 -0.94 -0.92 -0.90 -0.88 -0.86 -0.84 -0.82 -0.80 -0.78 -0.76 -0.74 -0.72 -0.70 -0.68 -0.66 -0.64 -0.62 -0.60 -0.58 -0.56 -0.54 -0.52 -0.50 -0.48 -0.46 -0.44 -0.42 -0.40 -0.38 -0.36 -0.34 -0.32 -0.30 -0.28 -0.26 -0.24 -0.22 -0.20 -0.18 -0.16 -0.14 -0.12 -0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86 0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 -3 0 3 Residuals

Variance:

rvf_plot(air_perf)
Fitted values 2.5 3.0 3.5 4.0 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 2.500 2.505 2.510 2.515 2.520 2.525 2.530 2.535 2.540 2.545 2.550 2.555 2.560 2.565 2.570 2.575 2.580 2.585 2.590 2.595 2.600 2.605 2.610 2.615 2.620 2.625 2.630 2.635 2.640 2.645 2.650 2.655 2.660 2.665 2.670 2.675 2.680 2.685 2.690 2.695 2.700 2.705 2.710 2.715 2.720 2.725 2.730 2.735 2.740 2.745 2.750 2.755 2.760 2.765 2.770 2.775 2.780 2.785 2.790 2.795 2.800 2.805 2.810 2.815 2.820 2.825 2.830 2.835 2.840 2.845 2.850 2.855 2.860 2.865 2.870 2.875 2.880 2.885 2.890 2.895 2.900 2.905 2.910 2.915 2.920 2.925 2.930 2.935 2.940 2.945 2.950 2.955 2.960 2.965 2.970 2.975 2.980 2.985 2.990 2.995 3.000 3.005 3.010 3.015 3.020 3.025 3.030 3.035 3.040 3.045 3.050 3.055 3.060 3.065 3.070 3.075 3.080 3.085 3.090 3.095 3.100 3.105 3.110 3.115 3.120 3.125 3.130 3.135 3.140 3.145 3.150 3.155 3.160 3.165 3.170 3.175 3.180 3.185 3.190 3.195 3.200 3.205 3.210 3.215 3.220 3.225 3.230 3.235 3.240 3.245 3.250 3.255 3.260 3.265 3.270 3.275 3.280 3.285 3.290 3.295 3.300 3.305 3.310 3.315 3.320 3.325 3.330 3.335 3.340 3.345 3.350 3.355 3.360 3.365 3.370 3.375 3.380 3.385 3.390 3.395 3.400 3.405 3.410 3.415 3.420 3.425 3.430 3.435 3.440 3.445 3.450 3.455 3.460 3.465 3.470 3.475 3.480 3.485 3.490 3.495 3.500 3.505 3.510 3.515 3.520 3.525 3.530 3.535 3.540 3.545 3.550 3.555 3.560 3.565 3.570 3.575 3.580 3.585 3.590 3.595 3.600 3.605 3.610 3.615 3.620 3.625 3.630 3.635 3.640 3.645 3.650 3.655 3.660 3.665 3.670 3.675 3.680 3.685 3.690 3.695 3.700 3.705 3.710 3.715 3.720 3.725 3.730 3.735 3.740 3.745 3.750 3.755 3.760 3.765 3.770 3.775 3.780 3.785 3.790 3.795 3.800 3.805 3.810 3.815 3.820 3.825 3.830 3.835 3.840 3.845 3.850 3.855 3.860 3.865 3.870 3.875 3.880 3.885 3.890 3.895 3.900 3.905 3.910 3.915 3.920 3.925 3.930 3.935 3.940 3.945 3.950 3.955 3.960 3.965 3.970 3.975 3.980 3.985 3.990 3.995 4.000 2 4 3.218794708351071-0.2941571966098067 3.218794708351071-0.4330983587443621 3.218794708351071-0.764512062526256 3.2187947083510710.2405383007862029 3.218794708351071-0.764512062526256 3.218794708351071-1.678580052759516 3.2187947083510710.4809696314888984 3.218794708351071-0.10985013836749188 3.218794708351071-0.862239828095601 3.218794708351071-0.5884213633031203 3.218794708351071-0.053725865907110676 3.218794708351071-0.862239828095601 3.218794708351071-0.4330983587443621 3.2187947083510710.8042178518657687 3.218794708351071-0.862239828095601 3.218794708351071-1.347166348977622 3.2187947083510710.1495559277070044 3.218794708351071-0.22981656513024643 3.2187947083510710.745598363227548 3.218794708351071-0.053725865907110676 3.218794708351071-0.22981656513024643 3.218794708351071-0.10985013836749188 3.218794708351071-0.2941571966098067 3.2187947083510710.3256466269301401 3.2187947083510710.8325785439888898 3.2187947083510711.703873443381044 3.2187947083510711.4132271701957728 3.2187947083510711.500591649766929 3.2187947083510711.7744101145594098 3.820750610305060.82011366377355 3.820750610305060.8045073117313211 3.820750610305061.2526982717819828 3.820750610305060.6725251174953555 3.820750610305060.6194150665908199 3.820750610305061.7185753019645815 3.82075061030506-0.018578305422740957 3.82075061030506-2.1409784525825746 3.82075061030506-1.0236286687351992 3.82075061030506-0.04821374037740491 3.82075061030506-0.5100331631847498 3.82075061030506-0.9036622419724447 3.820750610305060.3386302815487222 3.82075061030506-0.9622817306106647 3.820750610305060.46634821545927996 3.82075061030506-0.6442561758979485 3.82075061030506-0.04821374037740491 3.820750610305061.1601184165491698 3.820750610305060.8807552146886761 3.820750610305061.296659721139611 3.82075061030506-1.3822334669080731 3.82075061030506-2.1409784525825746 3.82075061030506-0.20728844407128447 3.8838344281456580.25544030370994003 3.8838344281456580.3627125357196649 3.8838344281456580.03717287887303219 3.8838344281456580.6895393779847909 3.8838344281456580.08889407092718474 3.883834428145658-1.1711592780535418 3.8838344281456581.0527310376744334 3.8838344281456580.6569767024129785 3.883834428145658-1.4654234447468553 3.8838344281456580.34194484105528794 3.8838344281456580.640388793677466 3.8838344281456580.2994017530675683 3.883834428145658-0.43318198704341715 3.883834428145658-0.01665995711758533 3.883834428145658-2.5555821342032514 3.883834428145658-0.7754049427920875 3.883834428145658-2.085227268286802 3.8838344281456580.7369236859347679 3.8838344281456580.9110736616927764 3.8838344281456580.9110736616927764 3.8838344281456580.6065735727939382 3.883834428145658-0.2570912878202814 3.8838344281456580.3627125357196649 3.883834428145658-0.5513554545135949 3.8838344281456580.010531137273393982 3.8838344281456581.346995204367747 3.236673355584195-0.8858167820280535 3.236673355584195-0.9913708100728236 3.236673355584195-0.3177341505422592 3.2366733555841950.49352428996573394 3.236673355584195-0.2533935190626989 3.236673355584195-0.13342709229994434 3.2366733555841950.562946705601216 3.2366733555841951.3530167117998126 3.2366733555841950.1722546466390201 2.82649414556310451.0344363742312481 2.82649414556310452.5298851391254042 2.82649414556310451.2925689285152737 2.82649414556310450.4074849919655698 2.82649414556310450.8429817572632019 2.8264941455631045-1.8992222134365793 2.8264941455631045-0.5652024869059107 2.8264941455631045-3.7273581939031 2.8264941455631045-0.5652024869059107 2.82649414556310450.7578734311192646 2.8264941455631045-1.3645267160405699 2.82649414556310450.9229287407849905 2.8264941455631045-0.2471769321931944 2.82649414556310450.08423677158869958 2.8264941455631045-0.2471769321931944 2.8264941455631045-0.5652024869059107 2.8264941455631045-0.07108623297005863 2.8264941455631045-0.985154223203319 2.82649414556310450.15553637096684905 2.82649414556310450.4074849919655698 2.82649414556310450.08423677158869958 2.8264941455631045-0.45045872580730945 2.82649414556310450.9983047618219602 2.82649414556310451.1698085995283274 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -4 -2 0 2 4 -4.0 -3.5 -3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 -4.00 -3.95 -3.90 -3.85 -3.80 -3.75 -3.70 -3.65 -3.60 -3.55 -3.50 -3.45 -3.40 -3.35 -3.30 -3.25 -3.20 -3.15 -3.10 -3.05 -3.00 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -2.60 -2.55 -2.50 -2.45 -2.40 -2.35 -2.30 -2.25 -2.20 -2.15 -2.10 -2.05 -2.00 -1.95 -1.90 -1.85 -1.80 -1.75 -1.70 -1.65 -1.60 -1.55 -1.50 -1.45 -1.40 -1.35 -1.30 -1.25 -1.20 -1.15 -1.10 -1.05 -1.00 -0.95 -0.90 -0.85 -0.80 -0.75 -0.70 -0.65 -0.60 -0.55 -0.50 -0.45 -0.40 -0.35 -0.30 -0.25 -0.20 -0.15 -0.10 -0.05 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 1.05 1.10 1.15 1.20 1.25 1.30 1.35 1.40 1.45 1.50 1.55 1.60 1.65 1.70 1.75 1.80 1.85 1.90 1.95 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00 -5 0 5 Std residuals
air_des = Dict(:Month => unique(air.Month))
air_eff = effects(air_des, model_air, invlink=exp)
5×5 DataFrame
Row Month log_oz err lower upper
String Float64 Float64 Float64 Float64
1 May 16.8862 2.66266 14.2235 19.5488
2 Jun 25.4489 6.55297 18.8959 32.0019
3 Jul 48.6103 7.3643 41.2459 55.9746
4 Aug 45.6385 7.35119 38.2873 52.9896
5 Sep 24.998 3.58588 21.4121 28.5839
effect_plot(
  air_eff,
  :Month, :log_oz,
  xlab = "Month",
  ylab = "Ozone (ppb)"
  )
Month May Jun Jul Aug Sep 524.99797216931815 445.638452159849905 348.61025068670836 225.448921263218626 116.886156522391964 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 0 10 20 30 40 50 60 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0 27.5 30.0 32.5 35.0 37.5 40.0 42.5 45.0 47.5 50.0 52.5 55.0 57.5 60.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8 11.0 11.2 11.4 11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4 15.6 15.8 16.0 16.2 16.4 16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4 19.6 19.8 20.0 20.2 20.4 20.6 20.8 21.0 21.2 21.4 21.6 21.8 22.0 22.2 22.4 22.6 22.8 23.0 23.2 23.4 23.6 23.8 24.0 24.2 24.4 24.6 24.8 25.0 25.2 25.4 25.6 25.8 26.0 26.2 26.4 26.6 26.8 27.0 27.2 27.4 27.6 27.8 28.0 28.2 28.4 28.6 28.8 29.0 29.2 29.4 29.6 29.8 30.0 30.2 30.4 30.6 30.8 31.0 31.2 31.4 31.6 31.8 32.0 32.2 32.4 32.6 32.8 33.0 33.2 33.4 33.6 33.8 34.0 34.2 34.4 34.6 34.8 35.0 35.2 35.4 35.6 35.8 36.0 36.2 36.4 36.6 36.8 37.0 37.2 37.4 37.6 37.8 38.0 38.2 38.4 38.6 38.8 39.0 39.2 39.4 39.6 39.8 40.0 40.2 40.4 40.6 40.8 41.0 41.2 41.4 41.6 41.8 42.0 42.2 42.4 42.6 42.8 43.0 43.2 43.4 43.6 43.8 44.0 44.2 44.4 44.6 44.8 45.0 45.2 45.4 45.6 45.8 46.0 46.2 46.4 46.6 46.8 47.0 47.2 47.4 47.6 47.8 48.0 48.2 48.4 48.6 48.8 49.0 49.2 49.4 49.6 49.8 50.0 50.2 50.4 50.6 50.8 51.0 51.2 51.4 51.6 51.8 52.0 52.2 52.4 52.6 52.8 53.0 53.2 53.4 53.6 53.8 54.0 54.2 54.4 54.6 54.8 55.0 55.2 55.4 55.6 55.8 56.0 56.2 56.4 56.6 56.8 57.0 57.2 57.4 57.6 57.8 58.0 58.2 58.4 58.6 58.8 59.0 59.2 59.4 59.6 59.8 60.0 0 100 Ozone (ppb)
Figure 9: Comparison of mean ozone concentrations by month. Error bars represent 95% CI around mean values.

4.5.2 Kruskal-Wallis test

Example

We are going to look at a RCT about treatment of children suffering from frequent and severe migraine.

fent = rcopy(R"pubh::Fentress")
fent |> schema
┌───────┬───────────────┬──────────────────────────────────┐
│ names │ scitypes      │ types                            │
├───────┼───────────────┼──────────────────────────────────┤
│ pain  │ Continuous    │ Float64                          │
│ group │ Multiclass{3} │ CategoricalValue{String, UInt32} │
└───────┴───────────────┴──────────────────────────────────┘
Exercise

Calculate statistics of pain by group from the fent dataset.

Code
pubh.estat(@formula(pain ~ group), data=fent) |> rcopy
3×9 DataFrame
Row group N Min Max Mean Median SD CV
String String Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 pain Untreated 6.0 -288.0 100.0 -55.0 -36.0 139.63 -2.54
2 Relaxation 6.0 43.0 100.0 84.0 96.0 23.3 0.28
3 Biofeedback 6.0 37.0 91.0 70.67 74.0 19.39 0.27
Exercise

Compare the mucociliary efficiency between groups with a strip chart.

fent_bst = pubh.gen_bst_df(
  @formula(pain ~ group), 
  data=fent
) |> rcopy
3×4 DataFrame
Row pain LowerCI UpperCI group
Float64 Float64 Float64 Cat…
1 -55.0 -162.78 36.0 Untreated
2 84.0 65.16 98.67 Relaxation
3 70.67 55.5 82.83 Biofeedback
Code
strip_error(
  fent,
  :group, :pain,
  xlab = "Cohort",
  ylab = "Pain reduction"
)
Cohort Untreated Relaxation Biofeedback 370.66666666666667 2,84.0 1,-55.0 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -300 -200 -100 0 100 200 -300 -280 -260 -240 -220 -200 -180 -160 -140 -120 -100 -80 -60 -40 -20 0 20 40 60 80 100 120 140 160 180 200 -300 -298 -296 -294 -292 -290 -288 -286 -284 -282 -280 -278 -276 -274 -272 -270 -268 -266 -264 -262 -260 -258 -256 -254 -252 -250 -248 -246 -244 -242 -240 -238 -236 -234 -232 -230 -228 -226 -224 -222 -220 -218 -216 -214 -212 -210 -208 -206 -204 -202 -200 -198 -196 -194 -192 -190 -188 -186 -184 -182 -180 -178 -176 -174 -172 -170 -168 -166 -164 -162 -160 -158 -156 -154 -152 -150 -148 -146 -144 -142 -140 -138 -136 -134 -132 -130 -128 -126 -124 -122 -120 -118 -116 -114 -112 -110 -108 -106 -104 -102 -100 -98 -96 -94 -92 -90 -88 -86 -84 -82 -80 -78 -76 -74 -72 -70 -68 -66 -64 -62 -60 -58 -56 -54 -52 -50 -48 -46 -44 -42 -40 -38 -36 -34 -32 -30 -28 -26 -24 -22 -20 -18 -16 -14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100 102 104 106 108 110 112 114 116 118 120 122 124 126 128 130 132 134 136 138 140 142 144 146 148 150 152 154 156 158 160 162 164 166 168 170 172 174 176 178 180 182 184 186 188 190 192 194 196 198 200 -300 0 300 Pain reduction
Figure 10: Comparison of mucociliary efficiency on pain reduction after dust removal.
Question

What is your main concern regarding your descriptive analysis?

Answer

Dispersion of pain reduction is greater in the Untreated group than in the other two groups. The sample size is relatively small for the control limit theorem to compensate.

We are going to perform the non-parametric, Kruskal-Wallis test, to test if the differences in pain reduction between groups are statistically significant or not.

KruskalWallisTest(
  vec_group(fent, :pain, :group)...
) |> pvalue |> r3
0.057
Interpretation

We did not find a significant difference between pain reduction in the untreated group and treatment groups (either relaxation or with biofeedback) (\(p=0.057\), Kruskal-Wallis test).